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ABSTRACT 


The Japanese maritime forces (Blue) are required to detect, identify and intercept 
maritime terrorist threats (Reds) well before they reach Japanese shores. However, it is 
challenging for the limited number of Blue maritime assets to identify and intercept Reds 
out of large numbers of law-abiding neutral vessels (Whites) within the limited time 
available to intercept a Red; there is a need to estimate Blue Maritime Intercept 
Operation (MIO) capabilities (a series of detection, identification and interception 
capabilities), and to identify the significant factors influencing the MIO capabilities 
quantitatively in order to examine current programs and to study new, alternative 


programs. 


This thesis formulates and exercises stochastic and simulation models to assess 
Blue MIO capabilities. The models focus on the surveillance operations of the Maritime 
Patrol Aircraft (MPA). The analysis using the models estimates the probability with 
which a Red is detected, correctly classified, and escorted for intensive investigation and 
neutralization before it leaves an area of interest (AOI). The difficulty of obtaining 
adequate interception of the Red depends upon the AOI size, the number of Whites in 
the AOI, detection and identification capabilities, information retention, and close 
coordination between the MPA and investigative maritime vessels in various situations. 
The analysis ultimately provides quantitative guidance on the relative importance of the 
MIO capabilities. Although the models focus on the MPA operations, the analysis 
additionally provides various insights and recommendations to other defense 


components. 
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EXECUTIVE SUMMARY 


The Japanese maritime forces (Blue) are required to detect, identify and intercept 
maritime terrorist threats (Reds) well before they reach Japanese shores. However, it is 
challenging for the limited number of Blue maritime assets to identify and intercept Reds 
out of large numbers of law-abiding neutral vessels (Whites) within the limited time 
available to intercept a Red; there is a need to estimate Blue Maritime Intercept Operation 
(MIO) capabilities (a series of detection, identification and interception capabilities), and 
to identify the significant factors influencing the MIO capabilities quantitatively in order 


to examine current programs and to study new alternative programs. 


In this study, we formulate and exercise stochastic and simulation models in order 
to assess Blue MIO capabilities. Logistic regression models are used to summarize the 
simulation output. The models focus on the surveillance operations of the Maritime 
Patrol Aircraft (MPA). There is one MPA to patrol an area of interest (AOI), otherwise 
called the Domain. One Red enters the AOI at time 0. The analysis using the models 
estimates the probability with which the Red is detected, correctly classified, and escorted 
for intensive investigation and neutralization before it leaves the AOI. The difficulty of 
obtaining adequate interception of the Red depends upon the AOI size, the number of 
Whites in the AOI, detection and identification capabilities, information retention, and 
close coordination between the MPA and investigative maritime vessels in various 


situations. 


The principal measures of effectiveness (MOEs) used to quantify the 


effectiveness of Blue MIO capabilities are 


e P.=Probability a typical Red is detected and correctly classified before leaving 


the AOI (Domain) 


e P,=Probability a typical Red is detected, correctly classified, and escorted before 


leaving the AOI (Domain) 
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Three types of models are used in this study: a simulation model, an analytical 
stochastic model, and logistic regression models. The analytical stochastic model is a 
special case of the simulation model. The MOEs for the stochastic model are expressed 
as closed-form formulas. The MOE formulas from the analytical stochastic model are 
used to check the output of the simulation in the special case. The formulas are also used 
to suggest useful independent variables for logistic regression models to summarize the 
simulation output in cases other than the special case. These independent variables 


include functions of the simulation input variables. 


In order to analyze the factors influencing Blue MIO capabilities, we eventually 


focus on the seven factors: (1) Dimensions of the rectangular AOI, M,andM, (2) 


Constant number of Whites in the AOI, w (3) Speed of Whites and Reds, u (4) 
Processing time for each contacted vessel by the MPA, 7 (5) Probability that a detected 


White is correctly classified as White, c,,, (Note that c,,=1—c,,, is the probability that a 


detected White is incorrectly classified as Red) (6) Probability that a detected Red is 


correctly classified as Red, c, (Note that c,,=1-—c,, is the probability that a detected 


Red is incorrectly classified as White), and (7) The MPA’s information retention (length 
of time for which the MPA retains the classification information on a vessel; the mean 
time is 1/y). For the MIO capabilities (P. and P,.), the results of the thesis identify the 
following tendencies: 

e When the AOI size (wu, respectively) increases, the MIO capabilities decrease. 


e When c,,, (c,,, respectively) increases, the MIO capabilities increase. 


e Changes to the value of 7 or 1/w do not change the MIO capabilities as much as 


changing the values of the other factors: the AOI size,w, u,c,,,, and c,. 


ww ? 


e There is an interaction between the AOI size, w, and c,,,. 


e w is the most influential factor for the MIO capabilities among the considered 


factors. 
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e Relatively high MIO capabilities can be achieved when there are a few Whites in 
a relatively small AOI; however, the MIO capabilities quickly decrease by 


increasing w a small amount (0O—10). 


For this analysis, we apply various assumptions (scenario, data, distributions, and 
modeling). If they are “reasonable,” it would imply that a single MPA is operationally 
inadequate to intercept a Red before it leaves an AOI. If more MPAs are 
simultaneously available, they should be used in the surveillance operation (MIO). 
Otherwise, measures which reduce the number of unidentified Whites in an AOI 
should be applied as much as possible — such measures include intelligence operations, 
maritime traffic control, and additional surveillance operations by other defense assets 


(satellites, helicopters, and maritime vessels). 


The results of the analysis cannot be directly applied to plan real concepts of 
operations (CONOPS) and operational plans (OPLAN) because the research is based on 
various assumptions. But they provide very useful intuition, enhancement, and 
stimulation. As a result, available field data should be collected to assess the 
reasonableness of the model assumptions. Although we focus on the MPA operations, the 
effects of other components (intelligence, helicopters, unmanned surveillance systems, 
maritime vessels, and C4I systems) can be studied using this MPA-based analysis. For 
example, reliable information of threatening vessels enables assignment of a small size 
AOI to an MPA, and surveillance operations by helicopters and maritime vessels can be 


used to complement the MPA’s surveillance operations. 
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I. INTRODUCTION 


Terrorism has become a serious threat to today’s global security environment — as 
demonstrated in the United States, Indonesia, Spain, Russia, and England from 2001 to 
2005. Japan has initiated several efforts to prevent terrorist attacks; however, its security 
is still vulnerable to threats (especially from the sea) resulting from its long coastlines, its 
large numbers of important facilities in coastal areas, and the large numbers of 


commercial ships and boats entering Japanese territorial waters. 


To protect Japan from maritime terrorist threats, the Japanese maritime forces 
(Blue) are required to detect, identify, and intercept the threats well before they reach 
Japanese shores. However, Blue has a finite number of assets (aircraft and vessels) used 
to execute a series of the operations called Maritime Intercept Operations (MIO). On the 
other hand, there are large numbers of vessels coming into or navigating in the waters 
around Japan, and terrorist vessels attempt to sneak into Japanese shores by camouflaging 
and hiding among these vessels. Thus, it is challenging for Blue maritime assets to detect, 
correctly classify, and intercept the threats out of the large numbers of law-abiding 


neutral vessels within the limited time available to intercept a terrorist vessel. 


The purpose of this thesis is to propose and analyze Blue surveillance, awareness, 


and neutralization capabilities to achieve Japanese maritime homeland security. 


A. CURRENT SITUATION 

For Japan’s maritime security during peacetime situations, the Japan Coast Guard 
(JCG), a police agency, is primarily in charge of the maritime security operations, and the 
Japan Maritime Self-Defense Force (JMSDF) does not have this responsibility under the 
existing law. If it is beyond the JCG’s power to perform maritime security operations, 


JMSDF shall take actions in a supporting position “in case it is particularly required.”! 


! Self-Defense Forces Law, Article 82 


However, as a result of the revision of the Self-Defense Forces Law, (which 
followed the incident of suspicious boats off Noto Peninsula in 1999), a step is taken that 
JMSDF assets (aircrafts and maritime ships) can be ordered into operation from the 
beginning of an emergency situation against serious maritime threats, such as suspected 
spy vessels, armed boats, and terrorist vessels.2 Moreover, since the possible intention of 
the maritime terrorist threats is to conduct military activities, JMSDF is being required to 


join maritime security operations against the threats more than ever. 


Thus, the framework of Japan’s maritime security is gradually changing to 
correspond with the current security environment. JMSDF and JCG created a manual for 
joint responses to suspicious ships in 1999, and have started several joint training 
exercises for security operations. However, the manual currently targets only relatively 
low-intensity situations (i.e. police operations) and restricts its focus to identifying 
respective responsibilities and establishing rules for information sharing. So, there are 
still no comprehensive concepts of operations (CONOPS) and operational plans 


(OPLAN) for the two agencies to jointly deter maritime terrorist threats in Japan. 


To effectively implement maritime security operations or the challenging MIO, 
appropriate CONOPS and OPLAN -— which include assets, systems, tactics, procedures, 
intelligence, and information sharing — are indispensable to the responsible maritime 
forces (JMSDF and JCG). Moreover, to enhance the MIO capabilities, the maritime 
forces are required to constantly examine current programs and to study new, alternative 
programs — such as system development, operational development, procurement, and 
training. Thus, it is essential for the forces to assess their own MIO capabilities in order 
to study and plan the proper CONOPS and OPLAN. It is also desirable for the forces to 
identify and assess the factors influencing the MIO capabilities to examine current 
programs and study new, alternative programs using the quantitative methods of 


operations research. 


2 Furusawa, Tadahiko, “On Territorial Defense — Policing Sea Area under Japanese Jurisdiction,” 
DRC annual report, (2002) 
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B. CURRENT STUDIES 

So far, several studies of maritime security operations and the MIO have been 
performed, focusing on various aspects of the operations. [Salcedo Franco, 1997] studies 
the resource (patrolling vessels) allocation for the maritime traffic control in Venezuela in 
order to maximize the number of “bad” vessels encountered. [Komiya, 2000] analyzes 
the surveillance route which maximizes the number of vessels detected and processed 
based on the previous flight information. [Nagai, 2003] examines the MIO’s decision- 
making process that was used to intercept the suspicious vessels that appeared off Amami 


Ohshima in 2001. 


However, the previous studies only focus on specific aspects of the maritime 
security operations, such as how to increase the number of vessels the maritime assets 
can process effectively. To properly assess the challenging MIO capabilities and to 
properly estimate the significant factors influencing the success of the MIO as a whole, 
we should consider a series of surveillance and intercept operations — such as detection, 


identification, and interception. 


C. OBJECTIVES 

This study formulates and exercises stochastic and simulation models to assess 
the likely MIO capabilities (a series of detection, identification, and interception 
capabilities). Specifically, we focus on the surveillance operations of the Maritime Patrol 
Aircraft (MPA). Using the models, the analysis estimates the probability with which a 
terrorist vessel is detected, correctly classified, and escorted for intensive investigation 
and neutralization before the vessel is able to leave an area of interest (AOI) and achieve 


a lethal effect. 


The difficulty of obtaining adequate interception of the hostile vessel depends 
upon the AOI size, the number of neutral vessels in the AOI, detection and identification 
capabilities, information retention, and close coordination between the MPA and 
investigating maritime vessels in various situations. The analysis ultimately provides 


quantitative guidance on the effectiveness of the MIO capabilities. 


D. STRUCTURE OF THESIS 

Chapter II describes the maritime intercept operations. Chapter III introduces the 
simulation model. An analytical stochastic model is formulated and studied. The 
analytical stochastic model is a special case of the simulation in that all random times are 
independent having exponential distributions. Formulas of the measures of effectiveness 
(MOEs) for the analytical stochastic model are presented. Results of a short study are 
displayed comparing the simulation output in this special case to that of the analytical 
stochastic model. Chapter IV presents the results from the simulation for cases in which 
the random times while independent are not necessarily exponentially distributed. 
Logistic regression is used to summarize the simulation output. The independent 
variables used in the logistic regression are suggested by the formulas from the analytical 
stochastic model. The independent variables include functions of the simulation input 
values. The operational implications of the models results are discussed. The thesis 
concludes with suggestions for further work. Appendices provide further details and 


model results. 


Il. SCENARIO DESCRIPTION 


A. JAPANESE FEATURES 

Japan is an island country surrounded by sea, having approximately 4,470,000 
square kilometers of territorial waters (territorial sea, contiguous zones, and economic 
exclusive zones) and 33,889 kilometers of coastlines. The oceans and maritime entries to 
Japan are sea routes for both good and bad influences. On the waters, commercial ships 
carry over 90 percent of the natural resources that are indispensable to Japanese 
existence. The same waters give illegal or asymmetric threatening vessels access to 
Japan. In the coastal areas of the Japanese islands, there are large numbers of important 
facilities — economic and military ports, atomic power plants, and oil refinery complexes. 
More than 11,000 commercial vessels come into Japanese ports from various foreign 
countries each month to access these facilities.3 There are also large numbers of 


domestic vessels and pleasure boats navigating in Japanese territorial waters. 





Figure 1. Area around Japan 


3 Ministry of Finance, “Entrance of Vessels by Nationality,” Trade Statistics of Japan, (2005) 
J 


B. TERRORIST THREATS 

Currently, there are several active terrorist groups: Al Qaeda, Jemaah Islamiyah, 
Basque Fatherland and Liberty (ETA), and Chechen separatists. Terrorists from these 
groups and others cause indiscriminate murders and subversive activities, both globally 
and regionally, in order to attain goals that are political, religious, or ideological in 
nature. In the vicinity of Japan, North Korean terrorist and guerrilla threats have become 
apparent through incidents such as the detection of suspicious boats appearing off Noto 
Peninsula in 1999 and off Amami Ohshima in 2001. 

Japan is one of the leading economic powers in the world and also has close 
relationships with the United States politically and economically. As a result, Al Qaeda 
has listed Japan as a target country. Thus, it is possible that these terrorists will attack 
Japan in order to promote their influence or message to Japan and the rest of the world. 
There are roughly two ways for terrorist threats to intrude into Japan: airplanes and ships. 
To destroy important infrastructures and populations crowded in coastal areas, some 
terrorists may try to sneak into Japan by sea, by camouflaging and hiding among large 


numbers of law-abiding neutral vessels. 


C. MARITIME INTERCEPT OPERATIONS 

Japan’s maritime security policy is to make its utmost efforts, by utilizing all 
available means, to prevent any threats from reaching Japan directly. To achieve this, 
several maritime assets (aircraft and maritime vessels) conduct constant surveillance, 
attempting to identify ships and submarines transporting guerillas and special operations 
units as early as possible in order to prevent them from advancing.4 5 For further support, 
Japan also consistently executes information sharing about possible threats with other 
nations and their intelligence agencies. 

The MIO’s surveillance operation by the Maritime Patrol aircraft (MPA) is the 
most vital in obtaining initial contact with threatening vessels, and is also the most vital 
in collecting information that can be used in the future by other defense assets. Thus, as a 


specific interest, we focus on the surveillance operations of the MPA of JMSDF. 


4 Japan Defense Agency, “National Defense Program Guideline for FY 2005 and After,” (2004) 
5 Japan Defense Agency, “Defense of Japan 2005 White Paper,” (2005) 
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JMSDF has several squadrons using MPAs, which are designated to respective 
regions to execute maritime defense operations. In each region, the MPA constantly 
executes surveillance operations over the area of interest (AOI). All contacts within the 
AOlI are tracked and identified. All available sensors (radar and visual) are used to detect, 
identify, and collect intelligence on contacts of interest. Every contact identified as a 
vessel is tracked, observed, and judged if it is a potential target. Maintaining accurate 
information on vessels previously processed is critical to prevent multiple interceptions 
of the same ships as they pass through specific areas.6 

If the MPA classifies a vessel as a target, it requests inspection units (maritime 
ships) to come and interrogate the vessel, and tracks the suspicious vessel until the 
inspection units reach the vessel to take over from the MPA. To keep from losing sight of 
the vessel, the MPA may not process other vessels while tracking the suspicious vessel. 
Tracking a real target is essential, but tracking a misclassified non-threatening vessel can 
impede the tracked vessel’s freedom of navigation, and also wastes time that is needed to 
detect and process real targets. Thus, it is important for the MPA coordinators to correctly 
identify each vessel contacted. After passing over the suspicious vessel to the inspection 
units, the MPA resumes the surveillance operations by returning to its flight route as soon 


as possible. 


anil, \ 
ane. 


Figure 2. Maritime Intercept Operations 


6Navy Tactical Support Activity, “Maritime Interdiction Force Procedures — Multi-National Maritime 
Manual,” (1996) 
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Hl. A MODEL OF MARITIME INTERCEPT OPERATIONS 


A. OBJECTIVE 

The objective of the model studied here is to estimate the Maritime Intercept 
Operations (MIO) capabilities — especially, the fraction of terrorist vessels that are 
detected, correctly classified, and escorted for intensive investigation and neutralization 
before the vessels leave an area of interest (AOI), otherwise called the Domain, and 
achieve a lethal effect. As a specific interest, we focus on the surveillance operations of 
the Maritime Patrol Aircraft (MPA). Obtaining adequate interception of hostiles depends 
upon the AOI size, the number of neutral vessels in the AOI, detection and identification 
capabilities, information retention, and close coordination between the MPA and 


investigative maritime vessels under various circumstances. 


B. MEASURES OF EFFECTIVENESS 
It is highly desirable for the Blue maritime assets to detect, identify, and intercept 
terrorist vessels before they leave an AOI. However, as a minimum, the MPA must detect 
and correctly classify hostile vessels before they pass through the AOI to Blue homeland 
(the coast of Japan). Thus, as measures of effectiveness (MOEs), we consider the 
following two measures: 
e Long-run fraction of terrorist vessels being detected and correctly classified 


before leaving the AOI 


e Long-run fraction of terrorist vessels being detected, correctly classified and 


escorted before leaving the AOI 


C. ASSUMPTIONS 

1. Intelligence 

Accurate information, intelligence, and knowledge of maritime terrorist threats — 
signs of activities, ship personnel, maneuvers, destinations, and possible areas where 


threat vessels exist — are important for the success of the MIO. However, intelligence is 


not always available or incomplete, and terrorists may attempt to execute surprise attacks. 
In this study, precise and timely intelligence is assumed not to be available; this is a 


worst-case scenario. 


2. Multi-Agency Operations 

To implement the challenging MIO as successfully as possible, close coordination 
between the Japan Maritime Self-Defense Force (JMSDF) and the Japan Coast Guard 
(JCG) is required. The coordination results in layered defense and information sharing 
between the two agencies. In this study, we focus only on the JMSDF operations — 
especially, the surveillance operation of the Maritime Patrol Aircraft (MPA). The MPA 
performs the most vital role in the MIO to obtain an initial contact of threatening vessels 


and to collect information for future use by other defense assets. 


Je JMSDF Operations 






Area of Interest 
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White vessel 
(Neutral) 
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Figure 3. | JMSDF operations 


10 


To implement the MIO, JMSDF organizes a Maritime Intercept Group (MIG) that 
is composed of the MPA and maritime vessels, and specifies an area of interest (AOJ) for 


surveillance far from Japanese shores. In this study, the following assumptions are made: 


The AOI is a rectangle. 
There is one MPA patrolling the AOI at a time. And, as a result of successive 
rotation of MPAs, there is constantly one MPA on patrol in the AOI. 

@ Maritime vessels also execute (barrier) patrol at the lower boundary of the 
AOI (nearest the Japanese coasts). However, in this thesis, it is assumed that 
they are entirely engaged in the inspection operation of the vessels classified 
as “suspicious” by the MPA. 

@ The patrolling MPA executes surveillance (here, random search) over the 
AOI. Whenever it detects an unknown vessel (usually by its radar), it 
approaches the vessel and judges if it is suspicious. It takes several minutes 
for the MPA to process (approach and classify) each contacted/detected 
unknown vessel. 

@ Times between detections for unknown vessels are independent random 
variables having exponential distributions. The mean time between detections 
depends on the following factors: 

- Size of the AOI 

- Surveillance speed of the MPA 

- Radar coverage or radar sweep width of the MPA 

- Time to process (approach and classify) each detected vessel 

- Number of unknown vessels in the AOI 

Since the mean time between detections for unknown vessels depends on the 
number of unknown vessels in the AOI, the times between detections are not 
identically distributed: they tend to be longer if unknown vessels are few, and 
shorter if there are many unknown vessels. 

@ The number of unknown vessels changes as they are classified and the 


classification information is retained. 
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The process (approach and classify) time for each detected vessel is assumed 
to be constant. 

If the MPA detects and classifies a vessel as “suspicious,” it tracks the vessel 
until the inspection unit (maritime ships) arrives to investigate and possibly 
escort the suspicious vessel. While tracking a suspicious vessel, the MPA 
cannot process other vessels. However, the MPA resumes the surveillance 
and the processing of other vessels after turning over the suspicious vessel to 
the inspection unit. Here we do not consider the possibility that the inspection 
units may be limited: there is always one available unit on station in the AOI. 
This is an optimistic assumption to be relaxed in future work. 

The random time for an inspection unit to relieve the MPA tracking a 


suspicious vessel has an arbitrary distribution F,,. Successive MPA tracking 


times are independent and identically distributed. In this study the spatial 
dependence of relief times is not explicitly considered. 

If an MPA detects and classifies a vessel as “non-suspicious,” it permits the 
vessel to pass through and resumes surveillance to detect and process other 
vessels. The MPA can retain the information (possible position and direction) 
of the vessels classified as non-suspicious for awhile. However, after some 
time passes, the MPA loses the classification information of the vessels. 
Thus, when the MPA detects the vessels somewhere again, the MPA may 
have to process them again. 

The random time for the MPA to lose the classification information of a 


classified vessel has an arbitrary distribution F,,. The times the MPA retains 


information for different vessels are assumed to be independent and 
identically distributed. 

The MPA has no learning capacity; that is, the MPA does not hold any 
information about a vessel after losing its classification information. 

The MPA tries to classify each vessel correctly. However, since its 
classification ability is not perfect, commercial vessels may be incorrectly 
classified as “suspicious.” In this case, the MPA is occupied by tracking a 


misclassified commercial vessel for awhile and wastes time. Conversely, the 
le 


terrorist vessels may be erroneously processed as “non-suspicious.” In this 
case, the MPA ignores the terrorist vessel crossing through the AOI until it 


next detects it as an unknown vessel. 


4. Terrorist Vessels 

In this thesis we call hostile terrorist vessels “Red vessels’, “Reds” or “Rs”. We 
assume that Rs operate independently; that is, Rs are assumed not to be executing joint 
operations. If an R is classified as “suspicious” and escorted by the maritime inspection 
units, it has been correctly identified as “threat.” After identified as “threat,” Rs may 
escape, destroy themselves, or attack inspection vessels. However, we do not consider 
such situations in this thesis. 

Specifically, we focus on the interval of time that starts when one R enters the 
AOI and ends when it is either escorted while crossing the AOI or it successfully passes 
through the AOI. We assume there is one R entering the AOI at time 0. The random time 


for an R to pass through the AOI has an arbitrary distribution F,, . All of the random times 


whose distributions are referred to above are assumed to be independent in this study. 


ro Commercial Vessels 

We call (non-hostile) commercial vessels “White vessels”, “Whites” or “Ws”. 
There are large numbers of Ws coming into, going out of, navigating, or staying in the 
AOI. Each W operates independently. Some Ws may be classified as “suspicious” and 
escorted by the maritime inspection units erroneously. In this thesis, we make the 
simplifying assumption that the total number of Ws in the AOI is constant. For more 


generality, see the working paper [Gaver, Jacobs, and Sato, 2005]’. 


At this point, we consider the legality of the MIO during peacetime situations. 
Subject to the United Nations Convention on the Law of the Sea, ships of all states, 
whether coastal or land-locked, enjoy the right of presumably innocent passage through 


the territorial sea.8 Thus, the maritime forces are not permitted to subjectively and 





7 Gaver, Donald P., Jacobs, Patricia A, and Sato, Hiroyuki, “Assessing resource requirements for 
maritime domain awareness and protection,” Working paper, Naval Postgraduate School (2005) 
8 United Nations Convention on the Law of the Sea, Article 17 
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haphazardly inspect a vessel with its domestic police law. However, to prevent the 
infringement of the customs, fiscal, immigration or sanitary laws and regulations of the 
coastal state, the coastal state can adopt laws and regulations that conform with the 
provisions of the convention and other rules of international law relating to innocent 
passage through the territorial sea.9 Here we assume that the maritime assets are 
permitted to escort any suspicious vessels for intensive investigation by a specific law or 


regulation under an emergency situation declared by the authorities. 


D. STRUCTURES 
1. Notation 
M,: Length of x-direction side of AOI (See Figure3), which corresponds to the 


length of homeland coastal region defended. 

M, : Length of y-direction side of AOI (See Figure3) 

v: Mean speed of the MPA 

v,: Mean speed of the inspection units (maritime vessels) 


u: Mean speed of Ws and Rs 

tT: Process (=approach and classify) time for each detected vessel 

f : Radar coverage or radar sweep width of the MPA 

C,,, . Probability that a W is correctly classified as W 

c,,,: Probability that a W is incorrectly classified as R (=1—c,,, ) 

c..: Probability that an R is correctly classified as R 

c,,,: Probability that an R is incorrectly classified as W (=1—c,, ) 

t,: Time at which the MPA finishes processing the j detected vessel 
W,(t,): The number of Ws unknown in the AOJ at timer, 


9 United Nations Convention on the Law of the Sea, Article 21 
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W,(¢,): The number of Ws identified as W in the AOI at timer, 

w: Total number of Ws in the AOI at time ¢, (w=W,(t,)+W,(t,)) 
R,(t,): The number of Rs unknown in the AOI at timer; 

R(t,): The number of Rs thought to be W in the AOI at timer, 

R(t,): Total number of Rs in the AOI at time rt; (R(t;)=R,(¢,)+ RC) 
N-(t,): Cumulative number of Rs correctly classified at time ¢, 

N,,(t,): Cumulative number of Rs being escorted at time f, 

N,(t,): Cumulative number of Rs passing through the AOI at time f, 
T..: Time at which an R is correctly classified before it leaves the AOI 
T,, : Time at which an R is escorted before it leaves the AOI 


1: If a W is correctly classified at time t, 
Tyc(t;,H) = and returns to be unknown at time t, + H 


0: otherwise 


1: If an R is incorrectly classified as W at time t, 
Tpc(t;,H) = and returns to be unknown at time t, + H 


0: otherwise 
L(t,) : Time until the next detection after time t 


U : Time an un-encountered R spends in the AOI 
D: Time the MPA tracking a suspicious vessel is occupied 
H: Time until the MPA loses classification information of a classified vessel 


O(t,): Detection rate at time f, 


1/4: Mean time an un-encountered R spends in the AOI 
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1/@: Mean time the MPA tracking a suspicious vessel is occupied 


1/w: Mean time until the MPA loses classification information of a classified 


vessel 


P..: Fraction of Rs being detected and correctly classified before leaving the AOI 


P,: Fraction of Rs being detected, correctly classified, and escorted before 


leaving the AOI 
2. Simulation Formulations 
a. Detection Rate 


Most models for target detection are for situations in which there are one 
or few possible targets.10 In the MIO situation, there are many possible targets, and each 
possible target takes time to process. For the Maritime Interdict/Intercept Operations or 
the Maritime Traffic Control situation, target detection (rate) is not principal, but is a 
valuable measure in representing the number of vessels processed in a period. There are 
few quantitative studies concerning the rate at which vessels are processed in an MIO 
setting. The model in Salcedo Franco [1997] treats the number of vessels processed by 


the maritime assets in a period (the processing rate) as constant. 


In this section, we formulate a detection rate model to handle the number 
of unknown vessels encountered and processed. The detection rate model d(t;) depends 
on the five variables: size of the AOI, surveillance speed of the MPA, radar coverage or 


radar sweep width of the MPA, processing (classification) time for each detected vessel, 


and the number of unknown vessels in the AOI. 


The MPA searches a rectangle region AOI with areaM,-M,. The MPA 


travels the AOI at a speedv. The radar coverage or radar footprint is assumed to be a 


square with sides of length f. The number of radar footprints necessary to cover the 


10 Landa Borges, Jose Manuel, “Radar Search and Detection with the CASA 212 S43 Aircraft,” Naval 
Postgraduate School, (2004) 
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whole AOI is (M 2M, )/ f° (see Figure4); the time for the MPA to cross one footprint 


area is f /v. Hence, the time for the MPA to transit the entire AOI (with no processing 


time) is 
























































M 


x 


Figure 4. | The number of footprints necessary to cover the whole AOI 


We assume that the positions of Ws are uniformly distributed over the 
AOI. Our model considers the time interval starting with the entrance of one R into the 
AOI until the R leaves the AOI; no other Rs enter the AOI. During this time interval, the 
number of (unknown) vessels the MPA expects to process while transiting the entire AOI 
is (roughly) w+1. The processing time of each detected vessel ist. Thus, the total time 
the MPA spends detecting (transiting) and processing vessels in the AOI during one pass 
through the AOI is 





[Ma Lawes (3.2) 


v 


17 


The long run average rate of target detections/classifications is 


wtl 


[re }La(wenye 
f v 





(3.3) 


Hence, detection rate at time is ¢ ; is taken to be 











5(t,)=-—__ et * Bi) gy 
“ (M,-M,\ f 
ME conene Ray) t 





If z =0 and there are always the same number of unknown vessels in the 
AOI, the model (3.4) is the common random search model!!. However, in this study, the 


detection rate d(t,) depends on 7 and the number of unknown vessels in the AOI at time 


t,. Thus, it is not constant and will vary over time. 


b. Mean Time an Un-encountered R Spends in the AOI 


We assume that an R travels along the y-direction of the AOI. The mean 


time an un-encountered R spends in the AOI is 














M, 
Vu=— (3.5) 
u 





Cc. Mean Time an MPA Tracking a Suspicious Vessel is Occupied 


We approximate the y -position in the AOI where a vessel is detected and 
classified as suspicious by M ,/2. After being classified as R, the vessel keeps advancing 
toward the Japanese shores (the bottom of the AOJ) until it meets the inspection maritime 
units. The x -position of the inspection units is assumed to be the same as that of the 


vessel. The relative speed the vessel and the inspection units approach isv, +u. The mean 


time an MPA tracks a suspicious vessel is taken to be 


11 Wagner, Daniel H., Mylander, Charles W., and Sanders, Thomas J., “Naval Operations Analysis, 3 
ed, ” Naval Institute Press, (1999) 
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M. 
/¢=—— | (3.6) 











d. Mean Time Until the MPA Loses Classification Information of a 
Classified Vessel 


The mean time until the MPA loses classification information depends on 
the MPA’s tactics, systems, human factors, C4ISR capabilities, and vessels behaviors. In 
this thesis the mean time until the MPA loses information concerning a previously 


classified vessel is treated as an external parameter. 


e. Fraction of Rs Being Detected and Correctly Classified Before 
Leaving the AOI 


The fraction of Rs being detected and correctly classified before leaving 


the AOI is taken to be 





___Nc@) 


C= NP) +N AT)’ T =min(T,, U) G.7) 


Below are further details of the simulation. We abbreviate the expression 
“with probability” as w.p. 

L(t :) ~ An exponential distribution F, with rate 0(¢,) 

U ~ An arbitrary distribution F,, with mean1/ 

D~ An arbitrary distribution F,, with mean1/@ 

H ~ An arbitrary distribution F,, with meanl/ywy 


All sample realizations of the above four random variables are mutually 





independent. 
ee ae LE Ry) 
ae _ : W,(t,)+R,(t,) ""  W,(t,)+R,(t,) 
i W,(t)) R,(t,) 


t,+L(t,)+D, wp. Ce 
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D, in the expression above is the MPA tracking time of the j classified 


vessel given the vessel is classified as a R; successive MPA tracking times are 


assume to be independent and identically distributed. 


W (t,) 
W (t,)-1+ I,-(t,,H,) w.p. ——+— 
w(t) pz we(t,»H;,) P W,(t,)+R,(1,) 
Ww, (ti) = tj Sty tg Stns 
W(t;) + >»; Iyc(t,,H,) otherwise 
eh ees 
W (t.) 
W.(t,) +1- [,(t,,H (5. 
i(t;) py welt,» H,) Pp W,(t,)+R,(t,) 
W, (ti) = tj <ty +H, St jy) 
Wit) - y Tyc(t,,H,) otherwise 
er em 
R (t,) 
R (t,)-1+ I,-(t,,H,) w.p. ——+—— 
w(t) pz rc (t,»H,) P W,(t,)+R,(t,) 
R, (ti) = tj <t +Hy St ja 
R,(t;) + by Tpc(t,,H,) otherwise 
tj ae Stat 
R (t,) 
R(t.) +1- CA wae 
i(t;) pz rc (ts H,) Pp W,(t,)+R,(t,) 
R(t 441) = tj <t +H St) 
RG) = » Ipc(t,,H,) otherwise 
tj ae, Stijn 
ae ee 
Toe» A) = W,(t,) +R, (4) 
0 otherwise 
R 
1 wp. aol a 
Tre (tysi» A) = W,(t,) +R, (G,) 


0 otherwise 


H, in the expressions above is the random time the MPA retains 


classification information concerning the k” vessel classified. The information 


retention times are assumed to be independent and identically distributed. 
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N.(t j= w.p. Wi)+RG) an if t.+ (4)< 
C\j+H uW'j uj 
0 otherwise 
1 if t,,>U and N-(t,,)=9 
Np (tin) = re eae (3.8) 
0 otherwise 
ee tin Uf Nc(tis) becomes | at time t,,, 
co ~— otherwise 
f Fraction of Rs being Detected, Correctly Classified, and Escorted 


before Leaving the AOI 


The fraction of Rs being detected, correctly classified, and escorted before 


leaving the AOI is taken to be 





N,(T) 


: T =min(T,, U) 
N,(T)+N,() 





E 





(3.9) 








This simulation is the same as the one above 


(Fraction of Rs being 


detected and correctly classified before leaving the AOI) except for using (3.10) 


instead of (3.8). 











R, (t;) fe 
WD Cone 

Ne(tj) = W(t) +R, (,) 
0 otherwise 
1 ift,,>U and N,(t,,,)=9 

Np (tj) = ne iar (3.10) 
0 otherwise 

as ti Uf N-(t,.,) becomes | at time t,,, 

7 co ~§=—- otherwise 
E. IMPLEMENTATION 


To implement the simulation formulations, we develop a discrete event simulation 


model using Simkit (initially developed by K. Stork!2 and subsequently reviewed and 


12 Stork, Kirk A., “Sensors in Object Oriented Discrete Event Simulation,” Naval Postgraduate 


School, (1996) 
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extended by Professor Arnold Buss!3, Naval Postgraduate School). Simkit has a number 
of pseudo-random generators. The default generator is a Mersenne Twister!4. 
Additionally, a variant of the Mersenne Twister is also included, as well as some other 
generators — SIMSCRIPT’s linear congruential generator (based on Peter Lewis' work15), 
a Tausworth generator, and several mixtures. The pooled random number generator in 
Simkit has a cycle length of approximately 2. The pooled generator guarantees that the 
cycle length of random numbers is the product of the two separate cycle lengths of the 
two underlying generators, as long as the two cycle lengths are relatively prime.!6 The 
following random variates are generated in Simkit according to the specified 


distributions. All random times are assumed to be independent. 


e L(t ) : Time until the next detection at time t j 


e U: Time an un-encountered R spends in the AOI 
e D: Time the MPA tracking a suspicious vessel is occupied 


e #: Time until the MPA loses classification information of a classified vessel 


F. VALIDATIONS 


We assume one R enters the AOI at time 0 and there is a constant number of Ws 


in the AOI, w. When the three distributions, F,, F,, and F,, are all exponential and the 


MPA has no memory of its classification information, the probability that a typical R is 


detected and correctly classified before the R leaves the AOI, P., and the probability that 


a typical R is detected, correctly classified, and escorted before the R leaves the AOI, P,., 


13 Buss, Arnold H., “Discrete Event Programming with Simkit,” Simulation News Europe, (2001) 


14 Matsumoto, Makoto and Nishimura, Takuji, “Mersenne Twister: A 623-Dimensionally 
Equidistributed Uniform Pseudo-Random Number Generator,” Keio University, (1998) 


15 Lewis, Peter A. W. and Learmonth, Gerard G., “Naval Postgraduate School random number 
generator package LLRANDOM,” Naval Postgraduate School, (1973) 


16 Hovda, Erik K., “A Simulation to determine the Effect that the Army Basic Officer Leadership 
Course will have on Accession Training,” Naval Postgraduate School, (2002) 
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can be calculated analytically.!7 To verify the simulation model, we compare the results 


of the simulation model with those of the analytical model. 


e F,: Distribution of the time between detections 
e F,,: Distribution of the time an un-encountered R spends in the AOI 


e F,,: Distribution of the time the MPA tracking a suspicious vessel is occupied 


1. Input Data 


As an example, we consider the following case as input data: 


M ,: Length of x-direction side of AOI (=400NM) 
M, : Length of y -direction side of AOI (=200NM) 


v: Mean speed of the MPA (=200kt) 

v,: Mean speed of the inspection units (=30kt) 

u: Mean speed of Ws and Rs (=15kt) 

tT: Process (=approach and classify) time for each detected vessel (=4min) 
f : Radar coverage or radar sweep width (=25NM) 


c,,,: Probability that a W is correctly classified as W (=0.95, 0.99) 


ww 


c,,, 1 Probability that a W is incorrectly classified as R (=1—c,,, ) 
c,, : Probability that an R is correctly classified as R (=0.6, 0.8) 
c,,, Probability that an R is incorrectly classified as W (=1—c,, ) 


w: Total number of Ws in the AOI (=100) 


L(t ;) ~ An exponential distribution F, with rate 6 (constant) 


17 Donald P. Gaver and Patricia A. Jacobs, “A Stochastic Modeling of a Variety of Simple MDA 
Situations,” (June), Naval Postgraduate School (2005). 
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M 
U ~ An exponential distribution F,, with mean 1/4 = — 
































u 
be ng Oh M,/2 
D~ An exponential distribution F,, with mean 1/¢ =— 
v,+u 
2; Derivation of the Measures of Evaluation (MOKs) for the Analytical 
Model 
a. Probability that one R is detected and correctly classified before 
leaving the AOI, P. 
—— rr 6c, 
P. = w+l 3 w+l _@ P. 
5c, +6 —c,, + bc, +5——c,, +n OTH 
wtl 1 wtl 1 
Solving 
bt, 
wtl 
a) rr ud wr Lt 
ps wt+l wt+l 
Cc 
5" ¢ 
1 w+l Y) 
5c, +6 c,, +4 OFM 
wl 1 
rT 
= w 
Pc= i a i (3.11) 
Ci Cup +H 
wl wt+l " @+yu 
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b. Probability that one R is detected, correctly classified, and escorted 
before leaving the AOI, P, 








baa, 
Poe wtl : Q 
ape ae ay 
wtl wtl 
5 ee 
w+l y) 





E 





52 ea Ot 
wtl wtl 





























Solving 
1 
d——c 
w+l ” y 
5c +6 6, +m [OTH 
P wtl wtl 
= 
Oe 
1 wei Y 
5c, +5" c,, +m |9TH 
wl +1 
Sy d 
P, = is a (3.12) 
1 w L 
a) Ch, Cwr +H 
wtl wtl @+u 











Simulation Implementations 


e For each case, 10,000 replications of the simulation are executed. 


e Each replication begins with a warm-up period. Specifically, about one 
week of operations (surveillance) is simulated before one R enters the AOI 
at time 0. Thus, the R may enter the AOI when the MPA is tracking a 


misclassified W. 
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4. Comparison (Analytical Model vs. Simulation Model) 


a. Probability that one R is detected and correctly classified before 


leaving the AOI, P. 












































C.. C. Simulation model 
as . Analytical 
Estimated 
Model 95%LB 95%UB 
Mean 
oe 0.6 0.1987 0.1976 0.1898 0.2054 
0.8 0.2485 0.2483 0.2398 0.2568 
i066 0.6 0.2451 0.2390 0.2306 0.2474 
0.8 0.3021 0.2951 0.2862 0.3040 
Table 1. Analytical results vs. simulation results for Pc 
Interval Plot of simulation results 
95% CI for the Mean 
0.35 
fea ft 
8 
£ 0.254 + hk 
£ 2 
s | | 
0.20 + 4 
Analytical 
Results 
0.15 ; . 1 1 
Crr 0.6 0.8 0.6 0.8 
Cww 0.95 0.99 


Figure 5. Analytical results vs. simulation results for Pc 


Since each analytical result falls in the 95% confidence interval of the 
estimated probability obtained from the simulation model, the results of the simulation 
model and the analytical model are not statistically/significantly different. Thus, we 


validate the simulation model to study the MIO capabilities ( PF. ) in various situations. 
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b. Probability that one R is detected, correctly classified and escorted 
before leaving the AOI, P, 














C.. C.. Simulation model 
ms " Analytical 
Estimated 
model 95%LB 95%UB 
Mean 
aoe 0.6 0.1703 0.1687 0.1614 0.1760 
0.8 0.2130 0.2129 0.2049 0.2209 
0.6 0.2101 0.2086 0.2006 0.2166 
0.99 





0.8 0.2590 0.2524 0.2439 0.2609 


























Table 2. Analytical results vs. simulation results for Pg 


Interval Plot of simulation results 
95% CI for the Mean 

















0.305 + 
0.25- 9 
- ss =z 
= A 
= 0.20- 
£ ae 
" A 
0.15- —— 
Analytical 
Results 
0.10 1 T T T 
Crr 0.6 0.8 0.6 0.8 
Cww 0.95 0.99 


Figure 6. Analytical results vs. simulation results for Pg 


Since each analytical result falls in the 95% confidence interval of the 
estimated probability obtained from the simulation model, the results of the simulation 
model and the analytical model are not statistically/significantly different. Thus, we 


validate the simulation model to study the MIO capabilities (P,) for a variety of 


situations. 
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IV. ANALYSIS 


Our interest is to estimate the results of likely Maritime Intercept Operations 
(MIO) capabilities (a series of detection, identification and interception capabilities) and 
to identify the significant factors influencing the results of MIO capabilities 
quantitatively. The MIO capabilities depend on several factors which are tactically 
controllable, systematically controllable, and uncontrollable. In this study, we consider 
the following factors: the size of an area of interest (AOI), the number of neutral vessels 
(Ws) in the AOI, detection and identification capabilities, information retention, and 


coordination between Maritime Patrol Aircraft (MPA) and maritime vessels. 


First, we (hypothetically) specify the factor values for each factor considered in 
this thesis (called Phasel). Next, we design an input data acquisition set by the Nearly- 
Orthogonal Latin Hypercube (NOLH) design, and implement the simulations in order to 
establish appropriate regression models of the MIO capabilities (called Phase2). Last, 
based on the regression models, we examine the impact of each factor simultaneously by 


changing the specified factor values (called Phase3). 


A. PHASE_1: FACTOR VALUES 

1. Area Size 

The size of the AOI is tactically controllable. It depends on the range of territorial 
waters, geographic features, the number of maritime assets (aircraft and maritime vessels) 


available, intelligence, and emergency levels. Here, we assume M, is 200NM as 


constant. M. is considered as being a value in the range 200NM-— 400NM. 


2. Mean Speed of the MPA 

We define the mean speed of the MPA, v, as the MPA’s transit speed between the 
location at which the MPA has processed a vessel and the next location the MPA contacts 
another unknown vessel. The maximum speed of the MPA is about 400kt. However, to 
process many vessels, the MPA is frequently forced to fly at low altitudes with a 


relatively low speed. Thus, we assume that v is 200kt and is constant. 


29 


3. Mean Speed of Inspection Units 
When required, the inspection units (maritime ships) should advance toward the 
possible area the suspicious vessel is in as soon as possible. Since the maximum speed of 


a maritime vessel is about 30kt, we assume v, is 30kt and is constant. 


4, Rader Sweep Width 

The radar sweep width depends on the radar performance, flight height, target 
strength, human factors, and environmental conditions.!8 We can estimate the radar 
sweep width in some specific conditions. In this section, we assume that f is estimated 
as 15NM and is constant during the MPA’s surveillance operations. This value may 


realistically change with atmospherics and altitude. 


5. Processing Time for Each Detected Vessel 

We define this processing time 7 as the time between when the MPA contacts an 
unknown vessel and when the MPA approaches and finishes classifying the vessel. Thus, 
tT depends on radar sweep width, density of vessels in the AOI, appearance of vessels, 


and operator’s skill. Using the values of f and v considered above, 7 can be considered 
as"(f /2)/v+classification time(= Q@) ~ 3(min)+a@".Thus, we assume that 7 is a 


constant value selected between 4min and 8min. 


6. Probability That a W is Correctly Classified as W 
The MPA operators have knowledge and experience about Ws navigating in 
Japanese territorial waters as a result of their constant surveillance operations. Thus, the 


probability of correctly classifying a White as W,c,,, may be estimated to be fairly high. 


ww ? 


However, some Ws may not have been detected and classified before, or may behave 


strangely or may be required to be inspected. Thus, we assumec,,, is constant with a 


value between 0.8 and 1.0. 


18 Frederickson, P. A. and K. L. Davidson, “An operational bulk evaporation duct model,” Working 
paper, Meteorology department, Naval Postgraduate School (2003) 
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cP Probability That an R is Correctly Classified as R 

The MPA operators have knowledge and experience about Ws. Thus, if the MPA 
operators find an R behaving unusually among other Ws, the MPA operators may easily 
classify the R as suspicious. However, Red vessels (Rs) may camouflage themselves 


cleverly, or some Ws may be hijacked by terrorist groups. Thus, we assumec,, is 


constant with a value between 0.6 and 1.0. 


8. Mean Time Until the MPA Loses Classification Information of a 
Classified Vessel 


The time until the MPA loses classification information depends on the MPA’s 
tactics, systems, human factors, C4ISR capabilities, and target behaviors. In this section, 
we consider 1/y to be a constant with a value between 0 hrs (the MPA has no memory of 
its classification information) and 4 hrs (each classified vessel is tracked by satellite or is 


held in a database or C4ISR systems). 


9. Mean Speed of Ws and Rs 
Based on the common speed of commercial vessels, we consider u as a constant 


with a value between 15kt and 30kt. 


10. Total Number of Ws in the AOI 

In this study, we make the simplifying assumption that the total number of Ws in 
the AOI is constant. We can estimate the approximate number of vessels in the AOI by 
the previous flight information and several statistical data. Furthermore, under emergency 
situations, the maritime forces may control maritime traffic (the number of vessels) in the 
AOI. Thus, total number of Ws in the AOI can be known approximately. Here, we 


assume w is aconstant with a value between 0 and 100. 


11. The Distribution of Time Between Detections 


The mean time between detections, which is the reciprocal of the detection 


rate 0(t,), is 
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(*: 2 \Fa(mareRen)s 

1 i v 

= (4.1) 
O(t;) W,(t,)+R,(;) 





In this study, we assume that the distribution of the times between detections ( F, ) 
is exponential. The times between detections are independent. The following figure 
displays the cumulative distribution function of F, whenM,=200NM M ,,=200NM, 


f =15NM, v=200kt, W,(t;)=100, R,(¢,)=1, and 7=4min. In this case, the mean time 
between detections is 


aos meee Ties ee 
1 15°) 200 


EW oe (A 
d(t,) 10041 ia 














Cumulative distribution function of F, 


























Time(min) 


Figure 7. | Cumulative distribution function of F 


The assumption that the times between detections have an exponential distribution 


with mean 12 minutes implies that about 2/3 of the times between detections are less than 
12 minutes. 
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12. The Distribution of Time an Un-encountered R Spends in the AOI 


In this study, we assume that the random time an encountered R spends in the 
eh ei ad : M, 

AOI has a Gamma distribution (F,, ) with mean 1/4 =—— and shape parameter f,, =25. 
u 


The following figure is the cumulative distribution function of F, when M,=200NM, 
u=20kt and f,,=25. In this case, the mean time an un-encountered R spends in the AOI 


iS 








M, _200NM 


l/u= 
‘s u 20kt 


=10hrs | (4.3) 











Cumulative distribution function of F, 





0.8 





0.6 


0.4 





0.2 











0.0 + 1 
0.0 10.0 20.0 30.0 
Time(hrs) 


Figure 8. Cumulative distribution function of Fu 


The figure implies that the time an encountered R spends in the AOI is a random 
value between Shrs and 15 hrs. When the time is as small as Shrs, we can consider the 
situation; the R grazes the AOI. On the other hand, when the time is as large as 15hrs, 
the R can be considered to have crossed the AOI diagonally. 
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13. The Distribution of Time the MPA Tracking a Suspicious Vessel is 
Occupied 


We assume that the random time an MPA tracking a suspicious vessel is occupied 


y 


has a Gamma distribution ( F,, ) with mean 1/¢@ = 





and shape parameter f,, =25. The 
v,+u 


successive tracking times are independent and identically distributed. The following 


figure displays the cumulative distribution function of F, when M = 200NM, v,=30kt, 


u=20kt, and f,, =25. In this case the mean time the MPA tracks a suspicious vessel is 





_M,/2_— 200NM/2 _ 
v, tu (30kt + 20Kt) 





(4.4) 











This is an approximate time for an inspection unit (maritime vessels) to meet the 


tracked suspicious vessel and release the MPA. 


Cumulative distribution function of Fp 





























Time(hrs) 


Figure 9. | Cumulative distribution function of Fp 
The figure implies that the time the MPA tracks a suspicious vessel is a random 
value between lhr and 3hrs. When the time is as small as lhr, we can consider the 
situation; a vessel is classified as R at a position close to the bottom of the AOI (near the 
Japanese shores). On the other hand, when the time is as large as 3hrs, a vessel is 


classified as R at a position far from the bottom of the AOI. 
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14. The Distribution of Time Until the MPA Loses Classification 
Information of a Classified Vessel 


We assume that the random time until the MPA loses classification information of 


a classified vessel has a Gamma distribution ( F,,) with mean 1/y and shape parameter 
£,,=50. The retention times for different classified vessels are independent and 
identically distributed random variables. The following figure displays the cumulative 


distribution function of F,, whenl/w=4hrs and £,,= 50. 


Cumulative distribution function of Fy, 


























Time(hrs) 


Figure 10. Cumulative distribution function of Fy 


The time until the MPA loses classification information of a classified vessel is a 
random value between 2.5hrs and 5.5hrs. The MPA may lose its classification 
information before the mean time 4.0hrs resulting from the MPA’s tactics and target 
behaviors. On the other hand, it is also possible that the information (possible position) of 
some vessels previously classified is available to the MPA for a long time by the renewal 
of the database (surface pictures) by other defense assets (Satellite, Automatic 
Identification System (AIS)!9 ). 


19 United States, Government Accountability Office, “Maritime Security: Partnering could reduce 
federal costs and facilitate implementation of automatic vessel identification system ” (2004) 
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B. PHASE _ 2: LOGISTIC REGRESSION MODELS 

In this phase, we describe and explore appropriate regression models of the MIO 
capabilities by designing the input data by the Nearly-Orthogonal Latin Hypercube 
(NOLH) designs and implementing the simulations. 


1. SIMULATION IMPLEMENTATION 
a. Multi-level Factor Values 


Table 3 shows the specified factor values of the multi-level factors. 






































Low High 
M. 200NM 400NM 
w 0 100 
u 15kt 30kt 
T 4min 8min 
Cy 0.8 1.0 
C,, 0.6 1:0 
l/y Ohrs Ahrs 
Table 3. Multi-level factor values 
b. Design of Experiment 


For the experiment involving less than 23 factors, NOLH designs are 
available for examining the impact on the simulated MOEs of simultaneously changing 
the specified factor values.20 Since the number of factors of this case is only 7 as shown 
in Table 3, we apply the NOLH to produce our basic design of experiments. However, 


the independent variables in the logistic regression models for PR. and P,, are obtained 


from the analytical Markov model.2! Details are in Appendix_1. 





20Kleijnen, Jack P.C., Sanchez, Susan M., Lucas, Thomas W., and Cioppa, Thomas M., “A User’s 
Guide to the Brave New World of Designing Simulation Experiments,” INFORMS Journal on 
Computing, (2005) 


21 Gaver, Donald P. and Jacobs, Patricia A., “A Stochastic Modeling of a Variety of Simple MDA 
Situations,” Naval Postgraduate School, (2005) 
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Table 4 displays the 34 design points. These design points are used as 
input to the simulations. Design points (#1 — 17) are obtained from the NOLH based on 
Table 3 using the NOLH design spreadsheet22. The spreadsheet uses the designs based on 
the Cioppa’s basic NOLH designs23. The Design points (#18 — 34) are intentionally 
added to obtain relatively higher probabilities for P. and P, by using the other NOLH 


based on Table 5. 


ID NM kt min hrs 
| =) © | = 2-) 2) 8 ==) nc] ee) 
Sa EE EE Ee ns EE eee ee) 
=) = =") | =] =] = =] =) 
ee) (2 8) = 8 0 =] =] = 0 A) =] 
ee) ==) ==] ==) =] ==) =] =) 
50 | 22.5] 


oO 
© 
om 
o/o 
oO 
Co} to 


( All 





| 300] 50 
| 336] 
| 250] 
| 200] 
| 06] 
| 250] 
| 225] 10 
| 234] 
| 2di] 


=o 


an 


Da 16. 0.94 


Table 4. Design of experiment 


22 Sanchez, Susan M., Sanchez, Paul J, Lucas, Thomas W., “NOLH designs spreadsheet,” Naval 
Postgraduate School,(2005). Available online via http://diana.cs.nps.navy.mil/SeedLab/ 





23 Cioppa, Thomas M. “Efficient Nearly Orthogonal and Space-filling Experimental Designs for High- 
dimensional Complex Models,” Naval Postgraduate School, (2002) 
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Low High 
‘ 200NM 250NM 
w 0 20 
u 15kt 20kt 
T 4min 6min 
Cas 0.95 1.0 
co 0.90 1.0 
l/y Ohrs 4hrs 





Table 5. Multi-level factor values additionally considered 


2; OUTPUT DATA 
a. Probability that the one R is detected and correctly classified before 


leaving the AOI, P. 


The probability P. is estimated as the fraction of replications for which the one R 
is detected and correctly classified before leaving the AOI. Table 6, which is sorted in 


descending order based on P., displays the outputs of the simulation model. Under the 
favorable situations (Design points #18 — 34), P. is estimated as in the range 0.40 to 0.58 
(Note: P. can not be more than about 0.6 because the design point #20 is nearly the most 


favorable situation for the Blue force). On the other hand, under the moderate situations 


(Design points #1 — 17), P. is estimated as in the range 0.10 to 0.37. 
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Design| Mx | My | W Vv Vi 
Point | (NM) | (NM) (kt) | (kt) 
ID 


| 15] 4.1] : 0.511 
| 2?7| 234] 200] oo] 200] 30 . . 0.508 
| 200] 30] : 0.498 


u f tau | Cww] Crr | 1/psi Simulation model 
(kt) | (NM) | (min) (hrs) 
| 15.3] 15] |_0.98) 


15.3 0.98 
0.5291 0.548 
0.530 
0.528 
0.518 
0.517 
0.508 
0.502 
0.491 
0.490 
0.479 
0.457 
0.463 
0.454 


| 200| 30] 0.497: 
209 0.488 
| 200| 30] 0.483 


| 200] 30] 
0.93] 2. 0.470 
200] 6] 200] 30] 16.9] 15] 5.6] 0.95] 0.98] 3. 0.460 

| 200] 30] 18.4] 15] 4.0] 0. 0.447" 
0.443 
0.434 
429 
0.422 
0.391 
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0.259 
0.236 
0.192 
0.194 


Gi/ho 
oD 
t3/O 


hoi 


| 34{ 200] 30) | 15] 7.3] 0.81] 0.178 
YS] 300] 200] 50) 200] 30) 0.177 
| at] 200] 30] 0.169 


| 63] 200] 30 0.1626 
| 8] 313] 200] 86] 200] 30} 26.3) 
| 6.5] 0.86] 
88 0. 





Table 6. Output of Pc 
b. Probability that the one R is detected, correctly classified, and escorted 
before leaving the AOI, Pr 


The probability P, is estimated as the fraction of replications for which the one R 


is detected, correctly classified, and escorted before leaving the AOI. Table 7, which is 


sorted in descending order based on P,., displays the outputs of the simulation model. 
Under the favorable situations (Design points #18 — 34), P, is estimated as in the range 
0.34 to 0.51 (Note: P, can not be more than about 0.53 because the design point #20 is 


nearly the most favorable situation for the Blue force). On the other hand, under the 


moderate situations (Design points #1 — 17), P, is estimated as in the range 0.08 to 0.32. 
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Design nea My Crr | 1épsi a model 
PeSMUBR SRN 

| Mean_| 95%LB | 95%UB | 

asoe| 5132 
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| 6] 400] 200) 
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Table 7. Output of Pg 





3: LOGISTIC REGRESSION MODELS 
a. Preliminary Study 


As a preliminary study, we explore appropriate logistic regression models 


for the following two cases. Details are in Appendix]. 


Notation: 


F, : Distribution of the time between detections 
F,, : Distribution of the time an un-encountered R spends in the AOI 


F,, : Distribution of the time for the MPA to track a suspicious vessel 
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CASE-1: 


e F, isan Exponential distribution with rate d(t p) 


e F, is a Gamma distribution with mean 1/u and shape | (i.e. Exponential 


distribution) 


e F, is a Gamma distribution with mean 1/@ and shape | (i.e. Exponential 


distribution) 


e The MPA has no memory of its previous classification information 
(pessimistic assumption) 


CASE-2: 


e F, is an Exponential distribution with rate d(t ip 
e F, isa Gamma distribution with mean 1/j and shape 8 (=5 to 50) 


e fF, isa Gamma distribution with mean 1/@ and shape @ which is the same as 


that of F,,.(U and Dare statistically independent.) 


e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


Our situation assumed in the Phase_1 is a specific case of CASE-2 except 


that the MPA has memory of its previous classification information. Thus, first, we 


explore appropriate logistic regression models which do not consider the MPA’s 


information retention (do not include the mean time until the MPA loses classification 


information of a classified vessel,1/ yw), based on the following statistically fitted logistic 


regression models (4.5) and (4.6) introduced in CASE2 of Appendix]; (this attempt is 


called “Not considering the MPA’s information retention”). After that, we explore 


appropriate logistic regression models which consider the MPA’s information retention; 


(this attempt is called “Considering the MPA’s information retention’’). 


Al 


e Probability that the one R is detected and correctly classified before leaving 
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This model is the same as (A.13) in Appendix]. 


(4.5) 


e Probability that the one R is detected, correctly classified, and escorted before 


leaving the AOI, P,, 
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This model is the same as (A.17) in Appendix]. 


(4.6) 


Note: The form of the independent/explanatory variables are derived from an 


analytical model. Simple use of the raw parameter values results in uselessly inferior 


predictions. See Appendix1. 
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b. Not Considering the MPA’s Information Retention 


To find appropriate logistic regression models which do not include the 


models are follows 


mean time until the MPA loses classification information of a classified vessel,1/w, we 
consider all independent variables of (4.5) (respectively (4.6)) except for In 8 (because 


we assume £ =25 as constant in the Phase-1). The statistically fitted logistic regression 


Probability that the one R is detected and correctly classified before leaving 


the AOI, P. 








nl = JrossieiaiTine, ~1.134Inu 


ne 

















w(l-c M wtl)t 
—0.440 In ( ww) +—~+ ( (4.7) 
2v,+3u fi-v M, 
(w+1)r)) 
wil-c.) M. wt+l)t 
+0.199} In (Cw) +—~+ 
2v,+3u fev M, 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 0.999 
R Square 0.998 
Adjusted R Square 0.998 
Standard Error 0.032 
Observations 34 
ANOVA, 
df SS MS fe Significance F 
Regression 4 18.581 4.645 4534 0.000 
Residual 29 0.030 0.001 
Total 33 18.611 
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
bO 0.881 0.177 4.984 0.000 0.519 1.242 
b1 1.117 0.046 24.089 0.000 1.022 1.212 
b2 -1.134 0.033 -34.722 0.000 -1.201 -1.068 
b3 -0.440 0.179 -2.460 0.020 -0.806 -0.074 
b4 0.199 0.045 4.429 0.000 0.107 0.290 





Since the R-square value of the regression model (4.7) is 0.998, and the p- 
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value of each independent variable is less than 0.05, the four independent variables of the 
model (4.7) are statistically significant in the logistic regression. Figure 11 and Figure 12 


display the outputs of the estimated regression model (4.7) and those of the simulation 


Logit [Estimated regression model (4.7) vs. Simulation model] 














Simulation model 




















Figure 11. 


Estimated regression model (4.7) 


Logit [Estimated regression model (4.7) vs. Simulation model] 


Pc [Estimated regression model (4.7) vs. Simulation model] 
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Estimated regression model (4.7) 


Figure 12. Pc [Estimated regression model (4.7) vs. Simulation model] 


the mean time 


Even though the estimated logistic regression model (4.7) does not include 


until the MPA loses classification information of a classified vessel,1/y, 


it summarizes the simulation output well for the input values considered. 
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e Probability that the one R is detected, correctly classified, and escorted before 


leaving the AOI, P,, 





























P 
n( £_|=],122Inc,, —1.474Inu +1.965In(2v, + 2u) 
EB 
M wtl 
—2.790In| c,, +wU-c,,,)+| —{+—-T |(2v, +3u) (4.8) 
3 M, 
2 
M wtl 
+0.269} In| c,, +wU-c,,,)+| —{+—-T |(2v, +3u) 
f:v M, 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 1.000 
R Square 1.000 
Adjusted R Square 0.965 
Standard Error 0.022 
Observations 34 
ANOVA 
df ss MS F Significance F 
Regression 5 51.477 10.295 22242 0.000 
Residual 29 0.013 0.000 
Total 34 51.490 
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
bO 0.000 - - - - - 
b1 1.122 0.031 35.804 0.000 1.058 1.186 
b2 -1.474 0.036 -40.780 0.000 -1.548 -1.400 
b3 1.965 0.063 31.403 0.000 1.837 2.093 
b4 -2.790 0.156 -17.828 0.000 -3.110 -2.470 
b5 0.269 0.027 9.896 0.000 0.214 0.325 


Since the R-square value of this regression model (4.8) is 1.000, and the p- 
value of each independent variable is 0, the five independent variables of the model (4.8) 
are statistically significant in the logistic regression model. Figure 13 and Figure 14 
display the outputs of the estimated regression model (4.8) and those of the simulation 


model. 
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Logit [Estimated regression model (4.8) vs. Simulation model] 
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Figure 13. Logit [Estimated regression model (4.8) vs. Simulation model] 


P; [Estimated regression model (4.8) vs. Simulation model] 
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Estimated regression model (4.8) 


Figure 14. P_ [Estimated regression model (4.8) vs. Simulation model] 


Even though the estimated logistic regression model (4.8) does not include 
the mean time until the MPA loses classification information of a classified vessel,1/y, 


it summarizes the simulation output well for the input values considered. 
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c. Considering the MPA’s Information Retention 

The logistic regression equations (4.7) and (4.8) do not have the mean 
time of information retention,1/w, as an independent variable. In Appendix 3, we 
consider logistic regression models which include it as an independent variable. For the 
logistic regression models and input values considered, the estimated coefficient of the 
mean time of information retention,1/y, is not statistically different from 0. Thus, the 


regression models (4.7) and (4.8) are used to summarize the simulation output. 


C; PHASE _3: ANALYSIS OF THE SIGNIFICANT FACTORS 
1. Significant Factors 


We initially consider seven multi-level factors (M,,w,u,T, c,,,, c,,, and l/y), 


however, 1/y is omitted because its estimated coefficients in the logistic regression 
models are not statistically different from 0 in Phase 2 (Details are in Appendix3). 
Since M ,=200NM, M, represents the size of the AOI. Table 8 displays the six significant 


factors that are analyzed in this phase and their values specified in Phase_1. 






































Low High 
M, 200NM 400NM 
w 0 100 
u 15kt 30kt 
3 4min 8min 
G.. 0.8 1.0 
Ca 0.6 1.0 
Table 8. Significant factors and specified values 


2. Logistic Regression Models of the MIO Capabilities 
We use the logistic regression models (4.7) and (4.8) to analyze the significant 
factors influencing the Maritime Intercept Operations (MIO) capabilities. In Appendix4, 


these models are cross-validated by other input values. 
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Js Data Analysis 

CASE-0: 

Initially, we consider a specific situation (M,=300NM, w=50, u=22.5kt, 7= 
6min, c,,,=0.9, and c,.=0.8). Each factor value is the middle value of the specified factor 
values in Table 8. This situation is called CASE-0. 


Probability that the one R is detected and correctly classified before leaving the 
AOI, P. 


The probability P. is estimated as “0.189” using the model (4.7). Fugure15 
displays the sensitivity of the probability P. by changing each factor value when other 


factors values are fixed. Figurel6 also displays the interaction of any two factors when 
other factor values are fixed. The value P.=0.189 is impractically low, and would be 
unacceptable operationally. The AOI size and the number of distracting Ws are simply 
too large for an unaided MPA to cover adequately. The Blue search capabilities must 


be enhanced in various ways. 
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Figure 15. Prediction Profiler (CASE-0: Pc) 
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Interaction Profiles 
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Figure 16. Interaction Profiler (CASE-0: Pc) 


The following tendencies are suggested by Figure 15 and 16: 


When M ,(w,u, respectively) increases, the probability P. decreases. 
When c,,,, (c,,, respectively) increases, the probability P. increases. 


Changes to the value of the mean time to classify a detected vessel, 7 , do not 


change P. as much as changing the values of the other factors. 
There is an interaction betweenM ,,w, and c,,,. 
w is the most influential factor for the probability PR. among the considered 


factors. 


A relatively high P. value can be achieved when there are a few Ws and a 
relatively small AOI; however, the P. value quickly decreases when w 


increases a small amount (0-10). 
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a. Probability that the one R is detected, correctly classified, and escorted 
before leaving the AOI, P, 


The probability P, is estimated as “0.150” using the model (4.8). Fugure17 


displays the sensitivity of the probability P, by changing each factor value when other 


factors values are fixed. Figure18 also displays the interaction of any two factors when 


other factor values are fixed. 
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Figure 17. Prediction Profiler (CASE-0: Pg) 
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Figure 18. Interaction Profiler (CASE-0: Pg) 
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The tendencies of the probability P, are quite similar to those of the probability 
P.. The second most influential factor for the probability P, can be identified as the 
velocity of the R, u. 

CASE-1: 

In CASE-1, we consider a specific situation [M ,=200NM, w=33, u=22.5kt, T= 
6min, c,,,=0.9, and c,=0.8]. w is the most influential factor in the regression models. 
To decrease w, we reduce the AOI size for an MPA. Here M , and w are assumed to be 
reduced to 2/3 of the CASE-0 values (M,=300NM—200NM, w=50-33). To 
implement this change, more MPAs or other surveillance assets are required in operation 
to cover the original size of the area (M,xM,=300NMx200NM); otherwise more 
reliable intelligence about the path of Rs is required to designate a specific AOI whose 
size is relatively small. For these parameter values, the probabilities P. and P, are 
estimated as 0.278 and 0.222 respectively. By reducing the AOI size by 2/3, P. 
(respectively P,) becomes 1.47 times [0.189-—-0.278] (respectively 1.48 times 


[0.150—+0.222]) of that of CASE-0. 
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Figure 19. Prediction Profiler (CASE-1: Pc) 
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Figure 20. Prediction Profiler (CASE-1: Pg) 
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CASE-2: 
In CASE-2, we consider a specific situation [M,=300NM, w=50, u=22.5kt, 7= 


6min, c,,,,=0.95, and c,,=0.90] in which the AOI size per an MPA can not be reduced 


(because additional MPAs or other surveillance assets are not available or because more 
specific intelligence about the path of Rs is not available): however, high identification 


capabilities cand c, are attainable using identification systems (database, experiences, 


and training.) Here we assume that c,,,=0.90—-0.95, c, =0.80—0.90. In this case, the 


probabilities P. and P, are estimated as 0.236 and 0.187 respectively. By enhancing the 
identification capabilities as mentioned above, P. (respectively P, ) becomes 1.25 times 
[0.189—+0.236] (respectively 1.25 times [0.150—0.187]) of that of CASE-0. However, 


the effect obtained by enhancing identification capabilities is smaller than that obtained 


by reducing the AOI size. Such low values are operationally unsatisfactory. 
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Figure 21. Prediction Profiler (CASE-2: Pc) 
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Figure 22. Prediction Profiler (CASE-2: Pg) 
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CASE-3: 
In Case-3, we consider a specific situation [M ,=200NM, w=33, u=22.5kt, T= 


6min, c,,,=0.95, and c,.=0.90] in which the changes in parameter values considered in 
CASE-1 and CASE-2 are applied simultaneously. In this case, the probabilities P, and 
P,are estimated as 0.343 and 0.274 respectively. By considering both measures 


(tactically reducing the AOI size per an MPA, and enhancing the identification 
capabilities), P. (respectively P, ) becomes 1.81 times [0.189—0.343] (respectively 1.83 


times [0.150—0.274]) of that of CASE-0. However, P.=0.343 and P,.=0.274 are still not 


operationally acceptable. 
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Figure 23. Prediction Profiler (CASE-3: Pc) 
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Figure 24. Prediction Profiler (CASE-3: Pg) 


For this analysis, we apply various assumptions (scenario, data, distributions, and 
modeling). If they are “reasonable,” it would imply that a single Maritime Patrol Aircraft 
(MPA) is operationally inadequate to intercept an R before it leaves an AOI. If more 
MPAs are simultaneously available, they should be used in the surveillance operation. 


Otherwise, measures which reduce the number of unidentified Ws in an AOI should be 
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applied as much as possible. These measures include intelligence operations, maritime 
traffic control, and additional surveillance operations by other defense assets (satellites, 


helicopters, and maritime vessels). 
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V. CONCLUSION AND FUTURE IMPROVEMENTS 


A. GENERAL 

This research formulates and exercises stochastic and simulation models to assess 
the Maritime Intercept Operations (MIO) capabilities and to quantitatively identify the 
significant factors influencing the MIO capabilities to detect and interdict hostile vessels. 
The research is based on various assumptions: scenario, data, distributions of the events, 
and models. It also focuses explicitly on the surveillance operations of the Maritime 
Patrol Aircraft (MPA) in the MIO. The model is also not spatial. Thus, the results of the 
analysis can not be directly applied to plan real concepts of operations (CONOPS) and 
operational plans (OPLAN). But they provide very useful intuition, enhancement, and 
stimulation. As a result, available field data should be collected to assess the 
reasonableness of the model assumptions. Although we focus on the MPA operations, the 
effects of other components (intelligence, helicopters, unmanned surveillance systems, 
maritime vessels, and C4I systems) can be studied using this MPA-based analysis. For 
example, reliable information concerning presence of threatening vessels enables 
assignment of smaller size of Area of Interest (AOI) to an MPA, and surveillance 
operations by helicopters and maritime vessels can be used to complement the MPA’s 


surveillance operations. 


B. CONCLUSIONS 

1. MIO Capabilities 

If our assumptions (scenario, data, distributions of the events, parameter values, 
and models) are “reasonable,” the probability that one R is detected and correctly 


classified before leaving the AOL P., is estimated as 0.189 (CASE-0) to 0.343 (CASE- 


3); the probability that one R is detected, correctly classified, and escorted before leaving 
the AOI,P., is estimated as 0.150 (CASE-0) to 0.274 (CASE-3). This would be 


ofr? 
operationally unacceptable and would imply that a single unassisted MPA is 
operationally inadequate. If more MPAs are simultaneously available, they should be 


used in the surveillance operation. Otherwise, the complementary surveillance 


SD 


operations by other defense components (intelligence, helicopters, maritime vessels, 


C4ISR systems) are surely indispensable for the success of the MIO capabilities. 
e CASE-0 (M ,=300NM, w=50, uv =22.5kt, t= 6min, c,,,=0.90, and c,,=0.80) 


° CASE-3 (M ,=200NM, w=33, u=22.5kt, 7= 6min, c,, =0.95, and c, =0.90) 


2. Factors Influencing the MIO Capabilities 

We initially consider more than ten factors that possibly influence the MIO 
capabilities. Since the length of y-direction side of the AOI, mean speed of the MPA, 
mean speed of Inspection Units, radar sweep width, and the distributions on the events 
(which are surely influential to the MIO capabilities) are assumed to be constant, we 
eventually consider the following seven factors which are important to evaluate current 


programs and study new, alternative programs. 
e w:The number of Ws in an AOI 


e M_.:Length of x-direction side of the AOI 


e u:Mean speed of Ws and Rs 


e 7:Process time for each detected vessel 


c,,,, : Probability that a W is correctly classified as W 


ww 


c,, :Probability that a R is correctly classified as R 


e 1/y:Mean time until the MPA loses classification information of a classified 
vessel. 
The most significant factor for the MIO capabilities is “w” that must be classified 
by one MPA. Relative high probabilities P. and P, can be achieved when there are few 


Ws (0 to 10) and a relative small AOI; however, the capabilities quickly decrease by 
increasing w to more than 10. Thus, for the success of the MIO, we should plan possible 


programs that enable restriction of w per one MPA. 
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e Increasing the number of the MPAs and reducing the number of unidentified 


Ws which should be processed by an MPA (i.e. reducing M, of the AOI per 
one MPA) 


e Enhancing intelligence operations and specifying a small AOI (or choke point 


area) where Rs probably pass through. 


e If traffic control is available for vessels entering Japanese territorial areas, it 


should be applied as often as possible. 


The mean speed of the Ws and Rs is probably the second most significant factor 
for the MIO capabilities. Thus, for Rs, transiting with a high speed may be a good way to 
intrude Japanese shores successfully. However, their unusual behavior among other Ws 
may make them easily identifiable as R using overhead sensors (satellites, the MPAs, and 


helicopters). 


The AOI size (here, represented by M, with a constant M ,=200NM) influences 


w which should be processed per one MPA. The large AOI forces an MPA not only to 
process more vessels and but also to take more time to cover the whole AOI. Thus, the 


AOI size can be considered as significant as w. 


The identification capabilities c,,,and c,, are also important factors to enhance 


the MIO capabilities: however, their effects are smaller than the programs to restrict w 
which should be processed by one MPA — such as increasing the number of the MPAs in 
operation and enhancing intelligence operations. Since it is usually difficult to procure 
more MPAs or to assign many MPAs simultaneously to surveillance operations, our 
interests may focus on enhancing the identification capabilities to deter Rs. However, 
since their effects are moderate, we should consider the cost effectiveness of procuring 


new identification systems or to applying new programs. 


In this study, the process time for each detected vessel 7 and the mean time until 
the MPA loses classification information of a classified vessel 1/y are identified as 


operationally insignificant factors. However, more time spent identifying the vessel may 


increase the MPA’s identification capability for an unknown vessel; this can enhance the 


ou 


MIO capabilities; see the working paper [Gaver, Jacobs, and Sato, 2005]24. Moreover, 
although information retention may not be significant for one MPA itself, it is important 
when we consider joint operations with other defense components for which information 


sharing is required. 


C. FUTURE IMPROVEMENTS 

The analysis provides several insights on the MIO capabilities even though we 
focus only on MPA surveillance operations. However, to support the planning of 
appropriate CONOPS and OPLAN practically, we must not only review our arbitrary 
assumptions (scenario, data, distributions and models) by analyzing available field data 
but we must also study the MIO capabilities more intensively by considering/ 
coordinating other defense components (intelligence, helicopters, maritime vessels and 
C4ISR systems) and developing more comprehensive large-scale models to study the 
missions/functions of the defense components, the coordination of the components, the 
information sharing among the components, and the significance of each component and 


its sub systems. 


24 Gaver, Donald P., Jacobs, Patricia A, and Sato, Hiroyuki, “Assessing resource requirements for 
maritime domain awareness and protection,” Working paper, Naval Postgraduate School (2005) 
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APPENDIX 1 LOGISTIC REGRESSION MODELS 


A. OBJECTIVE 
The objective of this appendix is to describe and explore appropriate logistic 
regression models for summarizing simulation output of the Maritime Intercept 


Operations (MIO) capabilities. 


B. MEASURES OF EFFECTIVENESS 
1. Probability that a typical R is detected and correctly classified before 


leaving the Area of Interest (AOI), P. 


2. Probability that a typical R is detected, correctly classified and escorted 
before leaving the AOI, P, 


GC, NOTATION 
M ,: Length of x-direction side of the AOI 


M,: Length of y-direction side of the AOI 

v: Mean speed of the Maritime Patrol Aircraft (MPA) 
v,: Mean speed of the inspective maritime vessels 

u: Mean speed of Ws and Rs 


tT: Process (=approach and identify) time for each detected vessel 


f : Radar coverage or radar sweep width 
c,,, : Probability that a W is correctly classified as W 


ww * 


c,,, 1 Probability that a W is incorrectly classified as R (=1—c,,, ) 


c,,: Probability that an R is correctly classified as R 


c,,, : Probability that an R is incorrectly classified as W (=1-c,,) 
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t,: Time at which the MPA finishes processing the j detected vessel 
W,(t,): The number of Ws unknown in the AOJ at timer, 

W,(¢,): The number of Ws identified as W in the AOI at timer, 

w: Total number of Ws in the AOI at timer, (w=W,(t,)+W,(t,)) 
R,(t,): The number of Rs unknown in the AOI at timer; 

R(t,): The number of Rs thought to be W in the AOI at timer, 

R(t,): Total number of Rs in the AOI at timer, (R(t;) = R,(t,) +R) ) 
L(t ;) : Time until the next detection at time f, 


U : Time an un-encountered R spends in the AOI 
D: Time the MPA tracking a suspicious vessel is occupied 


F, : Distribution of the time between detections 
F,, : Distribution of the time an un-encountered R spends in the AOI 
F,, : Distribution of the time for the MPA to track a suspicious vessel 


O(t;): Detections rate at time 1, 











O(t,)= Wi (3) + Bult) (A.1) 
i (M,-M,)f | 
7 JErtmare Rey) 











1/4: Mean time an un-encountered R spends in the AOI 











M. 
Wee! | AD) 
u 





1/@: Mean time for the MPA to track a suspicious vessel 
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CASE STUDIES 


In this appendix, we consider the following two cases: 


CASE-1: 


e F, is an Exponential distribution with rate d(t i) 


e F,, isa Gamma distribution with mean 1/u and shape | (i.e. Exponential 
distribution) 

e =F, is a Gamma distribution with mean 1/@ and shape 1 (i.e. Exponential 
distribution) 


e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


CASE-2: 


e F, is an Exponential distribution with rate d(t D) 
e F, isa Gamma distribution with mean 1/w and shape 8 (=5 to 50) 


e F, is a Gamma distribution with mean 1/¢@ and shape 8 which is the 


D 


same as that of F,,. (U and Dare statistically independent.) 
e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


MODEL IMPLEMENTATION 


Input parameters: 





1. Constant parameters 
M, =300NM, v=300kt, v, =30kt, f =15NM 
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2 Multi-level parameters 


For the following six parameters/factors, we consider the range between 


Low and High respectively. 



































Low High 
M, 50NM 300NM 

w 0 50 

u 10kt 30kt 

T 4min 8min 
C5 0.9 1.0 
C2 0.8 1.0 
Table 9. Multi-level parameters 


Design of experiments: 


For the experiment involving less than 23 factors, Nearly-Orthogonal Latin 
Hypercube (NOLH) designs are available for examining the impact on the simulated 


MOE when simultaneously changing the specified factor values.25 


Since the number of factors in CASE-1 is 6 (M,, w, u, 7, C,,,, C,,.) and the 


ww? 


number of factors in CASE-2 is 7(M,, w, u, 7, C,,,, C,,. B ), we apply the NOLH to 


produce our basic designs of experiment. However, the independent variables in the 
logistic regression models of P. andP, are those obtained for the logit of the 
probabilities obtained from the analytical Markov model.26 Details are in Appendix_2. 


The logit of the analytical Markovian model for P. (respectively P, ) appears in (A.4) 
(respectively (A.5)). 


In Be 
iP 


25 Kleijnen, Jack P.C., Sanchez, Susan M., Lucas, Thomas W., and Cioppa, Thomas M, “A User’s 
Guide to the Brave New World of Designing Simulation Experiment,’ INFORMS Journal on 
Computing (2005) 


26 Gaver, Donald P. and Jacobs, Patricia A., “A Stochastic Modeling of a Variety of Simple MDA 
Situations,” (June), Naval Postgraduate School (2005) 
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(A.4) 





wl-c..) M. wl 
=Inc,,—Inu-In ee aerate 7 i 
2v,+3u fv My, 

















In| “2 =Inc, —Inu+In(2v, +2u) 
“YE 
(A.5) 


M +1 
hn) a Se a ee Ou an) 
fv M 














CASE-1: 


Table 10 displays the 42 design points. These design points are used as 
input to the simulations. Design points (#1 — 34) are obtained from NOLH and design 
points (#35 — 42) are intentionally added to obtain higher probabilities for P. and P,.. 


For each design point, 10°=10,000 replications of the simulation are executed. 


Design Paint 
ID 





—. 





19 
3 
34 



































125 
12.5 




















Table 10. Design of experiment (CASE-1) 
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CASE-?: 


Table 11 displays the 42 design points. These design points are used as 


input data sets for the simulation. Design points (#1 — 34) are obtained by NOLH and 


design points (#35 — 42) are intentionally added to obtain higher probabilities for P. and 


P,.. For each design point, 10*=10,000 replications of the simulation are executed. 


Design Point] = Mx bee Beta 
ID NM Kt 


asf 8] | 


=== = SS 
31 5 te 3 ps0 a poe 96 pose 96 





ho} oo} ho 

oy olw 

my) O}c 
= 
mn 


nl —}— 
No 
P95) (a) es 





7) PT 7] 
-—_—17- 355 saa 87 of 0 a 8a] —— 
14 113 3,213 75] 097| 1.00 13 
a =) =) | 2) =] 2) 2 =] 
a =) EC) 0 (0) =] =D =) 
6d =) =] =) ec] =) =] ee) 
a =) 0) 0 A) 0) =) 2 ) 
9) ee) Ac) nc] ) 
20) =) =| =) =] A) 8] 
_—__20] 206] tf o st8t| et 
| TBAT 12.5] 53] 090] 0.96 
alas] 098] aa 
25 269 36, 28.8 73) 0.96 0.94 30 
=) 0) =] 0) Ae] =) 0] =) 
ed =) =) 72) ee) =) 2) ) 
| 28] S238] 18] BO] 7] 8B]? 
a) ed) 2) A) =) 


= SS eee 
a 2 275 B 5 00 0. 54 iG 





























Table 11. Design of experiment (CASE-2) 


DATA ANALYSIS 
CASE-1: 


1. Probability that the one R is detected and correctly classified before 
leaving the AOI, P. 
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Output data: 


The probability, P., is estimated as the fraction of replications for which 
an R is detected and correctly classified before leaving the AOI. Table 12 displays the 
outputs of the analytical Markov model (A.4) and those of the Exponential simulation 
model (all random times in the simulation are independent and have exponential 
distributions). Table 12 is sorted in descending order based on the outputs of the 
Exponential simulation model. The maximum value of P. is estimated as about 0.9. 


Design] Mx | My Vi Crr | Esementaleniulst ny simulation model 
Paint | (NM) } (NM) a (kt) «) Re hi Markov model Logit 
ID Lae ae TEST 

| 0.8980] 0.9096] 


36 3 ee 


| 0.8167 
0.7138) 0.7314 
0.7047| 0.7225 


7 


|_| 128] 300] 50] 
8} Ie sot _4at_ 300 
| 300] 


| 300] 4 
1] 300] 47| 300[ 30 15 0.2665| 0.2841 
[| 12) 269) 300) 28] 300) 30] 26.8] 15] 7.0] 0.94] 0.84] 0.2540] 0.2712 


Table 12. Analytical Markov model (A.4) and Exponential simulation model 
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Figure 25 and Figure 26 display the outputs of the analytical Markov 


model (A.4) and those of the Exponential simulation model. 


Logit [Analytical Markov model (A.4) vs. Exponential simulation model] 
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Exponential simulation model 
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Analytical Markov model (A.4) 


Figure 25. Logit [Analytical Markov model (A.4) vs. Exponential simulation model] 


Pc [Analytical Markov model (A.4) vs. Exponential simulation model] 
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Figure 26. Pc [Analytical Markov model (A.4) vs. Exponential simulation model] 
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Logistic regression model: 





We consider the following regression model based on (A.4) 














P 
C ae 
In =b, +b, Inc, +b, Inu 
TE 
' (A.6) 
w(l-c M wtl)t 
+ b; In Wl Cire)! fees ps SS 
2v,+3u fev M, 
e When b, #0 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 1.000 
R Square 1.000 
Adjusted R Square 1.000 
Standard Error 0.018 
Observations 42 
ANOVA 
df ss MS F Significance F 
Regression 3 35.681 11.894 38590 0.000 
Residual 38 0.012 0.000 
Total 41 35.693 
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
bO 0.040 0.032 1.234 0.225 -0.026 0.106 
b1 1.033 0.041 25,268 0.000 0.950 1.116 
b2 -1.009 0.008 -122.202 0.000 1.026 -0.993 
b3 -1.003 0.005 -208.788 0.000 -1.012 -0.993 


Since the p-value of b, is 0.225>0.05 and the 90% confidence interval 


contains 0. We next treat b, as 0. 


e When J, =0 


SUMMARY OUTPUT 


Regression Statistics 


MS 
12.925 
0.000 


F 
41383 


Significance F 
0.000 


Coefficients Standard Erro t Stat P-value Lower 95% Upper 95% 


Multiple R 1.000 

R Square 1.000 

Adjusted R Square 0.974 

Standard Error 0.018 

Observations 42 

ANOVA, 

af $s 

Regression 3 38.775 
Residual 39 0.012 
Total 42 38.787 
bO 0.000 - 
b1 1.025 0.041 
b2 -1.001 0.004 
b3 -1.007 0.003 
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25.224 
-237 064 
-313.404 


0.000 
0.000 
0.000 


0.943 
-1.009 
-1.013 


1.107 
-0.992 
-1.000 


Thus, the estimated logistic regression model is 








. P 
Estimated In c 
Cc 


(A.7) 
w(1—c,,,,.) A M, rm (w+l)-7 


=1.025Inc,, — 1.001 Inu —1.007 In 
2V,+3u f-v M 








y 








Since the estimated coefficients of the independent random variables are 
not statistically different than those of the Analytical Markov model (A.4), the estimated 
logistic regression model (A.7) is not statistically different from the Analytical Markov 
model (A.4). Table 13 displays the outputs of the estimated regression model (A.7) and 
those of the Exponential simulation model. Table 13 is sorted in descending order based 
on the outputs of the ica simulation model. 


Design] Mx | My Vi f | tau | Cww] Crr |—Szenentalermu ston simulation model_ 
Point | (NM) | (NM) ne (kt) (cy (NM) } (min regresion modelf Po 
ID LE Logit Eb aMean  95%LB 988 
| 15] 
ie 


3S 


0.5340 6 
0.5111} 0.5307 
[0.5089] 5 











Table 13. Estimated model (A.7) and Exponential simulation model 
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Figures 27 and Figure 28 display the outputs of the estimated regression 
model (A.7) and those of the Exponential simulation model. The estimated regression 


model (A.7) is not statistically different from the Exponential simulation model. 


Logit [Estimated regression model (A.7) vs. Exponential simulation model] 
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Figure 27. Logit [Estimated model (A.7) vs. Exponential simulation model] 


Pc [Estimated regression model (A.7) vs. Exponential simulation model] 
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Figure 28. Pc [Estimated model (A.7) vs. Exponential simulation model] 
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2 Probability that the one R is detected, correctly classified and escorted 
before leaving the AOI, P, 


Output data: 


The probability, P, , is estimated as the fraction of replications for which 


an R is detected, correctly classified, and escorted before leaving the AOI. Table 14 
displays the outputs of the analytical Markov model (A.5) and those of the Exponential 
simulation model (all random times in the simulation are independent and have 
exponential distributions). Table 14 is sorted in descending order based on the outputs of 


the Exponential simulation model. The maximum value of P, is estimated as about 0.8. 


Mx | My Crr Analytical eee simulation model 
(NM) | (NM) hs Pe fe ik pale Markov model Logit 
_ ist [wa [as 95%LB | 95%UB 
4 


| 35] 50] 300] of 300] 30] 10.0 -—15}_4.01 1.001 1.0 
a a 1 O| 1. 
| 50] 300] o] 300] 30) 
oof Oo} 3 


e242] 1 ba 
3 


34 
| 300] 6] 300) 
| 300] 0] 300) 
41 


41 0.36201 0.3708} 

3 44 O| 0.93] 0. 
0 3] 0.91] 0. 
|__ 41 


29{ 191] 300] 47] 300) 
Of 0.94] 0. 0.2203 


Table 14. Analytical Markov model (A.5) and Exponential simulation model 
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Figure 29 and Figure 30 display the outputs of the analytical Markov 


model (A.5) and those of the Exponential simulation model. 


Logit [Analytical Markov model (A.5) vs. Exponential simulation model] 
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Figure 29. Logit [Analytical Markov model (A.5) vs. Exponential simulation model] 
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Figure 30. Pg [Analytical Markov model (A.5) vs. Exponential simulation model] 
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Logistic regression model: 





We consider the following regression model based on (A.5) 




















P. 
E = 
In =b,+b,Inc,, +b, nu+b, In(2v, +2u)+ 
LE 
(A.8) 
M wtl 
x 
b, In| c,. +w(l-c,,,)+ +——.T |(2v, +3u) 
fv M 
y 
e When b, #0 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 1.000 
R Square 1.000 
Adjusted R Square 1.000 
Standard Error 0.015 
Observations 42 
ANOVA 
df Ss MS ia Significance F 
Regression 4 25.215 6.304 26306 0.000 
Residual a7 0.009 0.000 
Total 41 25.223 
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
bO -0.240 0.713 -0.337 0.738 -1.684 1.204 
b1 1.039 0.036 29,222 0.000 0.967 1.111 
b2 -1.025 0.075 -13.621 0.000 “1.177 -0.872 
b3 1.076 0.202 5.317 0.000 0.666 1.486 
b4 -1.007 0.005 -183.835 0.000 -1.018 -0.996 





Since the p-value of b, is 0.738>0.05 and the 90% confidence interval 


contains 0. We next treat b, as 0. 


e When b, =0 
SUMMARY OUTPUT 


Regression Statistics 


Multiple R 1.000 
R Square 1.000 
Adjusted R Square 0.973 
Standard Error 0.015 
Observations 42 
ANOVA 

df Ss: MS im Significance F 
Regression 4 25.831 6.458 27592 0.000 
Residual 38 0.009 0.000 
Total 42 25.839 

Coefficients Standard Erro: _t Stat P-value Lower 95% — Upper 95% 

bO 0.000 - - - - - 
b1 1.040 0.035 29.719 0.000 0.970 1.111 
b2 -0.999 0.010 -101.175 0.000 -1.019 -0.979 
b3 1.008 0.005 185.661 0.000 0.997 1.019 
b4 -1.007 0.005 -188.524 0.000 -1.018 -0.996 
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Thus, the estimated logistic regression model is 








Estimated In of; 
1-P 


E 


=1.040Inc,, — 0.999 Inu +1.008 In (2v, +2u) 
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ori e ayidi=e j2|2 =a 
fev 


+——-T 
M, 








(A.9) 


(2v, +3u) 








Since the estimated coefficients of the independent variables are not 


statistically different from those of the Analytical Markov model (A.5), the estimated 


regression model (A.9) is not statistically different from the Analytical Markov model 


(A.5). Table 15 displays the outputs of the estimated regression model (A.9) and those of 


the Exponential simulation model. Table 15 is sorted in descending order based on the 


outputs of the Exponential simulation model. 


Estimated 
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Exponential simulation model 
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Estimated model (A.9) and Exponential simulation model 


Figures 31 and Figure 32 display the outputs of the estimated regression 


model (A.9) and those of the Exponential simulation model. 


Logit [Estimated regression model (A.9) vs. Exponential simulation model] 
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Figure 31. Logit [Estimated model (A.9) vs. Exponential simulation model] 


P; [Estimated regression model (A.9) vs. Exponential simulation model] 
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Figure 32. P, [Estimated model (A.9) vs. Exponential simulation model] 
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CASE-2: 
e F, is an Exponential distribution with rate d(t *) 


e F, isa Gamma distribution with mean 1/” and shape @ (=5 to 50) 


e F, isa Gamma distribution with mean 1/@ and shape 8 which is same as 


that of F,,.(U and D are statistically independent.) 


e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


Input parameters: 





e Constant parameters 


M, =300NM, v=300kt, v, =30kt, f =15NM 


e Multi-level parameters 
Table 16 shows the specified range of the seven multi-level 
parameters/factors respectively. is the common shape parameter for the Gamma 
distributions F,, (distribution of the time an un-encountered R spends in the AOJ) and F,, 


(distribution of the time for the MPA to track a suspicious vessel). 



































Low High 
M, 50NM 300NM 
w 0 50 
u 10kt 30kt 
is 4min 8min 
Cs. 0.9 1.0 
C,, 0.8 1.0 
B 5 50 
Table 16. Multi-level parameters 


[= 





is Probability that the one R is detected and correctly classified before 
leaving the AOI, P. 


Output data: 


The probability, P., is estimated as the fraction of replications for which 
an R is detected and correctly classified before leaving the AOI. Table 17 displays the 
outputs of the Gamma simulation model. As a reference, the outputs of the analytical 
Markov model (A.4) (F,, and F,, follow exponential distribution) are also shown. Table 
17 is sorted in descending order based on the outputs of the Gamma simulation model. 


The maximum value of P. is estimated as almost 1.0. 


Design] Mx | My | we flv dv you f | tau | Cww] Crr ae —— Saran aeulatian Simulstion model 
Point | (NM) | (NM) (kt) | (kt) | (kt) | (NM) | (min) Markov model Logit 
ID tap es | Ween [Sob [Sa 
| | 1.00] 47| 1 
[soo oof sof roof a Hoot 
aso of ants 8 
a] : i 9 


Yd 
0.9865] 0.9907 


| 9 
3|_ 300] _3| 300) 


N}— 


|__ 16] 


| 0.5417] 0.5611] 





Table 17. Analytical Markov model (A.4) and Gamma simulation model 
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Figure 33 and Figure 34 display the outputs of the analytical Markov 


model (A.4) and those of the Gamma simulation model. 


Logit [Analytical Markov model (A.4) vs. Gamma simulation model] 
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Figure 33. Logit [Analytical Markov model (A.4) vs. Gamma simulation model] 


Pc [Analytical Markov model (A.4) vs. Gamma simulation model] 
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Figure 34. Pc [Analytical Markov model (A.4) vs. Gamma simulation model] 
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Logistic regression model: 





We consider the following regression model based on (A.4). 

















P 
In| —— |=), +5, Inc, +b, Inu+b, In B 
- P. 
’ (A.10) 
wil-c M wt+l)t 
+ b, In WIL= Cy) Epp ee fe he 
2v,+3u f-v M, 
e When b, #0 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 0.960 
R Square 0.922 
Adjusted R Square 0.913 
Standard Error 0.711 
Observations 42 
ANOVA 
df 33 MS F Significance F 
Regression 4 220.163 55.041 109 0.000 
Residual 37 18.730 0.506 
Total 41 238.893 
Coefficients Standard Erro: t Stat P-value Lower 95% Upper 95% 
bO -0.575 1.396 -0.412 0.683 -3.403 2.254 
b1 3.932 1.657 2.374 0.023 0.576 7.289 
b2 -2.244 0.335 -6.700 0.000 -2.923 -1.565 
b3 0.237 0.166 1.431 0.161 -0.099 0.574 
b4 -2.501 0.195 -12.843 0.000 -2.895 -2.106 


Since the p-value of b, is 0.683>0.05 and the 90% confidence interval 
contains 0, we next treat b, as 0. 

e When b, =0 

SUMMARY OUTPUT 


Regression Statistics 


Multiple R 0.970 
R Square 0.942 
Adjusted R Square 0.911 
Standard Error 0.704 
Observations 42 
ANOVA, 

af ss MS iz Significance F 
Regression 4 303.486 75.871 153 0.000 
Residual 38 18.816 0.495 
Total 42 322.301 

Coefficients Standard Error it Stat P-value Lower 95% Upper 95% 

bO 0.000 - . - - - 
b1 4.035 1.620 2.492 0.017 0.757 7.314 
b2 -2.354 0.199 -11.841 0.000 -2.757 -1.952 
b3 0.215 0.155 1.388 0.173 -0.098 0.528 
b4 -2.445 0.139 -17.594 0.000 -2.726 -2.164 
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Although the p-value of b, is 0.173>0.05 and the 90% confidence interval 


contains 0, here we hold the variable In # in the regression model. The estimated logistic 


regression model is 








= 4.035Inc,, —2.354Inu+0.215In B 


(A.11) 





— 2.445 In 


WL Coe) M, _ (wrt 
2v,+3u  f-v M 
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Table 18 displays the outputs of the estimated regression model (A.11) 


and those of the Gamma simulation model. Table 18 is sorted in descending order based 


on the aac of the Gamma simulation model. 
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Table 18. Estimated regression model (A.11) and Gamma simulation model 
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Figure 35 and Figure 36 display the outputs of the estimated regression 


model (A.11) and those of the Gamma simulation model. 


Logit [Estimated regression model (A.11) vs. Gamma simulation model] 
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Figure 35. Logit [Estimated regression model (A.11) vs. Gamma simulation model] 


Pc [Estimated regression model (A.11) vs. Gamma simulation model] 
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Figure 36. Pc [Estimated regression model (A.11) vs. Gamma simulation model] 
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Here we still consider the regression model which includes quadratic terms. 








In He =b, +b, Inc,, +b, Inut+b, ln B 
ine 

- +1)t 
w= Cw), Me , (wt) 


+b, In 
2v,+3u fv M 
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The statistically fitted quadratic regression model, which includes linear and 


quadratic terms is 








oe a J-13.2797+ 2.119%, 1.773100 +0.287Inf 
~#E 

WL Cid) M, _(wtlt 
2V p43 fe M 


+7.530In 





(A.13) 
: 

- +1 
+1.510} In WO) Cw) 4 Me pcre Ms 
2v,+3u fie M 


y 








SUMMARY OUTPUT 


Regression Statistics 








Multiple R 0.985 
R Square 0.971 
Adjusted R Square 0.967 
Standard Error 0.438 
Observations 42 
ANOVA 

of ss MS id Significance F 
Regression 5 231.991 46.398 242 0.000 
Residual 36 6.902 0.192 
Total 41 238.893 

Coefficients Standard Error it Stat P-value Lower 95% Upper 95% 

bO 13.737 2.015 6.819 0.000 9.651 17.823 
b1 2.119 1.045 2.027 0.050 -0.001 4.239 
b2 1.773 0.215 -6.260 0.000 -2.209 -1.338 
b3 0.287 0.102 2.801 0.008 0.079 0.494 
b4 7.530 1.283 5.871 0.000 4.929 10.132 


bS 1.510 0.192 7.054 0.000 1.120 1.901 


Although the p-value of b, is 0.05 and the 90% confidence interval barely 


contains 0, we treat the variable Inc,, as a significant factor in the regression model. 
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Table 19 displays the outputs of the estimated quadratic regression model (A.13) 


and those of the Gamma simulation model. Table 19 is sorted in descending order based 


on the outputs of the Gamma simulation model. 


Design| Mx | My Vi Crr | Beta Estimated }-— Sang ottion camaulanen model 
Paint | (NM) | (NM) Ps (kt) «) aA (min) regression modell Pe SS sLoogit 
ID aH est aan 95%UB 
Ot 


|__ 36] 
ee 
ea 


of | 300] 30 
| 41 _75_ 00] _of_soot_ao}_128 
|__| 


| 50] 300] 9 


O 
| 6] 300] 300] 16] 300] 30] 17.5] 
[7] 206|300| 9| 300] 30] 300] 
of Oo} 3 0 





Table 19. Estimated regression model (A.13) and Gamma simulation model 
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Figure 37 and Figure 38 display the outputs of the estimated quadratic regression 


model (A.13) and those of the Gamma simulation model. Although, at each design point, 


the difference between the output of the estimated regression model (A.13) and that of 


the Gamma simulation model is moderate, the estimated regression model (A.13) 


summarizes the output of the Gamma simulation model well. 


Figure 37. 


Figure 38. 


Logit [Estimated regression model (A.13) vs. Gamma simulation model] 
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Logit [Estimated regression model (A.13) vs. Gamma simulation model] 


Pc [Estimated regression model (A.13) vs. Gamma simulation model] 
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Estimated regression model (A.13) 


Pc [Estimated regression model (A.13) vs. Gamma simulation model] 
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2. Probability that the one R is detected, correctly classified, and escorted 
before leaving the AOI, P, 


Output data: 


The probability, P, , is estimated as the fraction of replications for which 


an R is detected, correctly classified, and escorted before leaving the AOI. Table 20 
displays the outputs of the Gamma simulation model. As a reference, the outputs of the 


analytical Markov model (A.5) (F,, and F, follow exponential distribution) are also 


shown. Table 20 is sorted in descending order based on the outputs of the Gamma 


simulation model. The maximum value of P, is estimated as almost 1.0. 


Design| Mx | My | W Vv Vi u f tau | Cww] Crr | Beta Analytical a ae simulation model 
Point | (NM) | (NM) (kt) | (kt) | (kt) | (NM) | (min) Logit 
ID Se ee ean ee ee 
0 4-3 2 4 47 0.99! 000 
9 
sot 0 


0.9959] 0.9951 
aoe a 





Table 20. Analytical Markov model (A.5) and Gamma simulation model 
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Figure 39 and Figure 40 display the outputs of the analytical Markov 


model (A.5) and those of the Gamma simulation model. 


Logit [Analytical Markov model (A.5) vs. Gamma simulation model] 
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Figure 39. Logit [Analytical Markov model (A.5) vs. Gamma simulation model] 


P; [Analytical Markov model (A.5) vs. Gamma simulation model] 
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Figure 40. Pg [Analytical Markov model (A.5) vs. Gamma simulation model] 
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Logistic regression model: 





We consider the following regression model based on (A.5) 





In , =b,+b,Inc,, +b, nu+b,In(2v,+2u)+b, in B 

















E 
(A.14) 
M. wil 
+b, In| c,, +w(-c,,,)+| —=+——-t |(2v, +3u) 
fv M 
y 
e When b, #0 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 0.948 
R Square 0.899 
Adjusted R Square 0.885 
Standard Error 0.619 
Observations 42 
ANOVA 
of $s MS F Significance F 
Regression 5 214.727 42.945 64 0.000 
Residual 36 24.166 0.671 
Total 41 238.893 
Coefficients Standard Erro:__t Stat P-value Lower 95% Upper 95% 

bO -68.608 37.889 -1.811 0.079 -145.449 6.234 
b1 5.020 1.683 2.666 0.011 1.202 8.639 
b2 -9.508 3.996 -2.380 0.023 -17.612 -1.405 
b3 22.324 10.743 2.078 0.045 0.535 44.113 
b4 0.263 0.192 1.372 0.179 -0.126 0.652 
bd -2.925 0.290 -10.076 0.000 -3.514 -2.337 


Since the p-value of b, is 0.079>0.05 and the 90% confidence interval 
contains 0, we next treat b, as 0. 


e When b, =0 
SUMMARY OUTPUT 


Regression Statistics 








Multiple R 0.958 
R Square 0.918 
Adjusted R Square 0.662 
Standard Error 0.844 
Observations 42 
ANOVA 
df ss MS in Significance F 
Regression 5 295.934 59.187 83 0.000 
Residual 37 26.368 0.713 
Tatal 42 322.301 


Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
b1 5.328 1.932 2.758 0.009 1.414 9.243 


b2 -2.337 0.547 -4.275 0.000 “3.445 -1.229 
b3 2.679 0.342 6.409 0.000 2.185 a:573 
b4 0.230 0.197 1.169 0.250 -0.169 0.628 
b5 -3.012 0.295 -10.210 0.000 -3.610 -2.414 
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Although the p-value of b, is 0.25>0.05 and the 90% confidence interval contains 
0, here we hold the variable Inf in the regression model. The estimated logistic 


regression model is 





we. 


In = 5,328 Inc, —2.337 Inu +2.879In(2v, + 2u)+0.230In B 


‘ (A.15) 
—3.012In} c, +w(l-c,,,)+ Lee Pes (2v, +3u) 
f:v M 


y 














Table 21 displays the outputs of the estimated regression model (A.15) and those 
of the Gamma simulation model. Table 21 is sorted in descending order based on the 
outputs of the Gamma simulation model. 


Mx | My | Woy Vv f tau | Cww} Crr | Bet Estimated Garnma simulation model 
(NM) | (NIM) (kt) (NM) | (rnin) regression model Pe Logit 
F Mean | 95%LB | 95%UB 
9992 
4.0/ 1.00 


| so} 300] of 300] 30] 100] 15] 40] 1.00] 1.00] 47| 0.9992] 1.0000] 
1 0 i 5.7 9991 | 





a 
19 


Vi u 
(kt) | (kt) 
| 4 10.0] 15] 4.0] 1.00) 

















15 


























Table 21. Estimated regression model (A.15) and Gamma simulation model 
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Figure 41 and Figure 42 display the outputs of the estimated regression model 


(A.15) and those of the Gamma simulation model. 


Logit [Estimated regression model (A.15) vs. Gamma simulation model] 
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Figure 41. Logit [Estimated regression model (A.15) vs. Gamma simulation model] 


P; [Estimated regression model (A.15) vs. Gamma simulation model] 
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Figure 42. Pr, [Estimated regression model (A.15) vs. Gamma simulation model] 
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Here we still consider the regression model which includes quadratic terms. 
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(A.16) 


The statistically fitted quadratic regression model, which includes linear and 


quadratic terms is 





P. 





E 


n( 
1 


}- 1.871Inc,, —2.689 Inu +3.911In(2v, + 2u)+0.302In B 


M wtl 


—9.810In| c,, mtn) +| Me tty, +3u) 


fy 


y 


M 





a 


wtl 


y 


+2.128} In] c,, rte +| ett ela, +3u) 


2 








SUMMARY OUTPUT 


Regression Statistics 


(A.17) 


Multiple R 0.985 
R Square 0.970 
Adjusted R Square 0.938 
Standard Error 0.447 
Observations 42 
ANOVA 
df 83 MS F Significance F 
Regression 6 230.656 38.443 192 0.000 
Residual 36 7.189 0.200 
Total 42 237.846 


Coefficients Standard Erro: t Stat P-value Lower 95% Upper 95% 
0.000 


bO - - . - - 
bt 1.871 1.073 1.745 0.090 -0.304 4.047 
b2 -2.689 0.292 -9.204 0.000 -3.282 -2.097 
b3 3.911 0.237 16.503 0.000 3.430 4.3591 
b4 0.302 0.104 2.694 0.006 0.030 0.513 
bd 9.610 0.665 -11.347 0.000 -11.563 -6.056 
b6 2.128 0.253 6.413 0.000 1.615 2.641 


Although the p-value of b, is 0.09>0.05 and the 90% confidence interval contains 


0, we treat the variable Inc, as a significant factor in the regression model. 
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Table 22 displays the outputs of the estimated quadratic regression model (A.17) 
and those of the Gamma simulation model. Table 22 is sorted in descending order based 


on the outputs of the Gamma simulation model. 


Mx | My Crr | Beta Estimated a ee simulation model 
(NM) | (NM) fe he K) Re hina Logit 
Besgarse 
a 


a 


| 9 
| 6 


mE 
8 6 





Table 22. Estimated regression model (A.17) and Gamma simulation model 
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Figure 43 and Figure 44 display the outputs of the estimated quadratic regression 
model (A.17) and those of the Gamma simulation model. Although, at each design point, 
the difference between the output of the estimated regression model (A.17) and that of 
the Gamma simulation model is moderate, the estimated regression model (A.17) 


summarizes the output of the Gamma simulation model well. 


Logit [Estimated regression model (A.17) vs. Gamma simulation model] 
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Figure 43. Logit [Estimated regression model (A.17) vs. Gamma simulation model] 
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Figure 44. P; [Estimated regression model (A.17) vs. Gamma simulation model] 
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G. DISCUSSION AND CONCLUSION 


CASE-1: 


e F, is an Exponential distribution with rate d(t A) 


e F, is a Gamma distribution with mean 1/u and shape | (i.e. Exponential 


distribution) 


e F, is a Gamma distribution with mean 1/@ and shape 1 (i.e. Exponential 


distribution) 


e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


Tables 12 — 15 and the corresponding graphs show that if an all Markov model (in 
which all random variables are independent and exponentially distributed) is simulated, 
compared numerically to a corresponding analytical/numerical-calculated Markov model, 
and compared to a statistically fitted logistic regression model based on the explanatory 


variables/parameters suggested by the Markov model, the models’ results for both P. 
and P,are very close for the parameter values selected. However, the highest P,, values 


are approximately 0.8, and these are achieved only when there are a few (~0 or 1) 


Whites in the area of interest (AOI) and c,,, ~1.0 and c,, ~1.0. A large number of 


ww 


Whites (w =25-50) and only slightly less high classification capabilities c,,, and c,, 
(=0.90-0.95) reduces P. and P, quickly, to approximately 0.25—0.50. This would be 


operationally unacceptable; so if the all Markov model were “reasonable,” it would imply 
that a single Maritime Patrol Aircraft (MPA) is operationally inadequate, especially if it 
has no effective memory of previous identifications, and/or is not effectively supported 
(by an overhead/satellite sensor system), and does not have warning intelligence. The 
number of unidentified vessels in the AOI may be decreased by a highly effective 


Automatic Identification System (AIS)27 coverage. 


27 United States, Government Accountability Office, “Maritime Security: Partnering could reduce 
federal costs and facilitate implementation of automatic vessel identification system ” (2004) 
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CASE-2: 


e F, isan Exponential distribution with rate d(t p) 
e F, isa Gamma distribution with mean 1/w and shape 8 (=5 to 50) 


e F, isa Gamma distribution with mean 1/@ and shape 8 which is the same as 


that of F,,.(U and Dare statistically independent.) 


e The MPA has no memory of its previous classification information 


(pessimistic assumption) 


Examination of Tables 18, 19, 21, and 22 along with the corresponding graphs 
(Simulation model, Analytical/numerical-calculated Markov model, and statistically 
fitted linear logistic regression applied to simulated data) show that the results for P. and 
P, differ considerably between the models. The Markov model (in which all random 


variables are independent and exponentially distributed) is pessimistic when compared to 


the more physically plausible gamma model with shape parameter / (approximately 25 
or more). For example, for design point #39 with shape parameter f =5 (not large), the 
simple Markov model gives P, ~0.76, while the simulated gamma model gives P, ~0.96 


(which is possibly unrealistically high). However, when a statistically fitted quadratic 


regression model is considered, the results for P. and P, for the estimated logistic 


regression model are close to those of the gamma simulation model for design point #39; 


the estimated regression model gives P, ~0.98, while the simulated gamma model 


gives P, ~0.96), as shown in Tables 19 and 22, and the corresponding graphical figures. 
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APPENDIX 2 LOGIT OF ANALYTICAL MARKOVIAN MODELS 


1. Probability that the one R is detected and correctly classified before 


leaving the AOI, P. 
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2s Probability that the one R is detected, correctly classified and escorted 


before leaving the AOI, P, 
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APPENDIX 3 LOGISTIC REGRESSION MODELS (PART2) 


A. OBJECTIVE 
The objective of this appendix is to describe and explore appropriate logistic 
regression models for the Maritime Intercept Operations (MIO) capabilities which 


consider the effects of Maritime Patrol Aircraft (MPA)’s information retention. 


B. MEASURES OF EFFECTIVENESS 
1. Probability that the one R is detected and correctly classified before 


leaving the Area of Interest (AOI), P. 


2. Probability that the one R is detected, correctly classified and escorted 
before leaving the AOI, P, 


C. PREVIOUS STUDIES 
The statistically fitted logistic regression models which do not consider the effects 
of the MPA’s information retention (do not include the mean time until the MPA loses 


classification information of a classified vessel,1/y) are (A.18) and (A.19) introduced as 


(4.7) and (4.8) in the Chapter IV. 


1. Probability that the one R is detected and correctly classified before 
leaving the Area of Interest (AOI), P. 
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Dr. 


Probability that the one R is detected, correctly classified and escorted 


before leaving the AOI, P, 
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D. 
1. 


INPUT DATA 


Constant parameters 


M,=200NM, v=200kt, v, =30kt, f =15NM 


Multi-level parameters 


(A.19) 


Table 23 shows the specified factor values of the multi-level factors. 
































Low High 
M. 200NM 400NM 
w 0 100 
u 15kt 30kt 
(A 4min 8min 
Cn, 0.8 1.0 
Gs 0.6 1.0 
l/y Ohrs Ahrs 











Table 23. 


Multi-level factor values 


Design of experiment 


Table 24 displays the 34 design points. These design points are used as 


input to the simulations. Design points (#1 — 17) are obtained from the Nearly- 


Orthogonal Latin Hypercube (NOLH) based on Table 23, and design points (#18 — 34) 
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are intentionally added to obtain relatively higher probabilities for P. and P, by using 


the other NOLH based on Table 25. 


ID NM kt min hrs 
|_____3|_226|_4at_1891_6.0}__0.83|_0.83,__a.0 
350 94 21. e 0.86 0.60 3.3 
staal ita et 
Se ee a ae 
aS | ae aa: a es ee ee ee) 
| 9 0 2] 0) 090] 0] 2.0) 
a |) a a 2: eS 
SE ee aE ers en EE en ee) 
a) 0) =) ec | =) =) | ee) =) 
31 15.0 6.8 0.83 0.83 0.3 
a :) |) 2) |) =| A) Ee) 
| CT TC “$C, '§_— 153] TB] 8] 4 
| (=) ES) EP =) =) =) 
2) 28 | 19} — a | 0 | 90] 3.3) 
Sa BE a EE aa ee ee 
PT ttt Tt] a3 
|_____26|_228|__18|__18.al__6.91__0.89|_0.98/__2. 
27 234 Oo 15.9 23 0.99 0. 31 1. 8 
ae: a a a ee ee ee 
ee) | a | dT de) 
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SS Se a ee ee en ee ee) 


Table 24. Design of experiment 


















































Low High 
M, 200NM 250NM 
w 0 20 
u 15kt 20kt 
€ 4min 6min 
Co, 0.95 1.0 
ex 0.90 1.0 
l/y Ohrs 4hrs 





Table 25. Multi-level factor values additionally considered 
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LOGISTIC REGRESSION MODELS 


1. Probability that the one R is detected and correctly classified before 


leaving the Area of Interest (AOI), P. 


We consider the following two types of logistic regression models as alternatives. 


e 1/y is considered as a single independent variable. 
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(A.20) 


When the fifth independent variable (the mean time until the MPA loses 


classification information of a classified vessel, 1/w) is 0, the model (A.20) corresponds 


to the model (A.18) which does not consider l/y. 


1/y influences the number of White vessels in the AOI, w. 
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(A.21) 


When the mean time until the MPA loses classification information of a 


classified vessel, 1/w is 0 or n is large, the model (A.21) corresponds to the model 
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(A.18) which does not include 1/w. n is determined so that the four independent 


variables are statistically significant in this regression model. 


The statistically fitted logistic regression models: 


e 1/y is considered as a single independent variable 
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(A.22) 








SUMMARY OUTPUT 


Regression Statistics 


F Significance F 
3631 0.000 


P-value Lower 95% Upper 


95% 





Multiple R 0.999 
R Square 0.998 
Adjusted R Square 0.998 
Standard Error 0.032 
Observations 34 
ANOVA, 
af SS MS 

Regression 5 18.582 3.716 
Residual 28 0.029 0.001 
Total 33 18.611 

Coefficients Standard Error t Stat 
bO 0.886 0.177 5.013 
b1 1.116 0.046 24.063 
b2 -1.133 0.033 -34.670 
b3 -0.420 0.180 -2.335 
b4 0.204 0.045 4517 
b5 0.005 0.005 1.012 





0.000 0.524 
0.000 1.021 
0.000 -1.200 
0.027 -0.788 
0.000 0.111 
0.520 -0.005 


1.248 
1.211 


-1.066 
-0.051 


0.296 
0.014 


Since the p-value of b, is 0.32>0.05 and the 90% confidence interval contains 0, 


b, is not statistically different from 0. Figure 45 and Figure 46 display the outputs of the 


regression model (A.22) and those of the simulation model. 
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Logit [Estimated regression model (A.22) vs. Simulation model] 
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Figure 45. Logit [Estimated regression model (A.22) vs. Simulation model] 
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Figure 46. Pc [Estimated regression model (A.22) vs. Simulation model] 


The estimated regression model (A.22) summarizes the output of the simulation 
model well. However, we employ the estimated model (A.18) rather than the estimated 
model (A.22) because the fifth independent variable of the model (A.22) is not 


statistically significant in the regression model. 
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e 1/y influences the number of White vessels in the AOI, w. 








ol ~ Jos90+1 16h, —1135In4 
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(A.23) 








+ 
2v,+3u pv M, 
SUMMARY OUTPUT 
Regression Statistics 
Multiple R 0.999 
R Square 0.998 
Adjusted R Square 0.998 
Standard Error 0.034 
Observations 34 
ANOVA 
df Ss MS F Significance F 
Regression 4 18.577 4644 3980 0.000 
Residual 29 0.034 0.001 
Total 33 18.611 
Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 
bO 0.890 0.193 4.607 0.000 0.495 1.285 
b1 1.116 0.050 22.531 0.000 1.015 1.217 
b2 -1.135 0.035 -32.495 0.000 -1.206 -1.064 
b3 -0.407 0.197 -2.071 0.047 -0.809 -0.005 
b4 0.210 0.049 4.276 0.000 0.110 0.310 


Since the R-square value of the regression model (A.23) is 0.998, and the p-value 


of each independent variable is less than 0.05, the estimated model (A.23) summarizes 


the simulation model output well. When n is less than 30, we could not find an 


appropriate logistic regression model which satisfies a high R-square value and the 


significance of the four independent variables. Figure 47 and Figure 48 display the 


outputs of the regression model (A.23) and those of the simulation model. 
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Logit [Estimated regression model (A.23) vs. Simulation model] 
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Figure 47. Logit [Estimated regression model (A.23) vs. Simulation model] 


Pc [Estimated regression model (A.23) vs. Simulation model] 
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Figure 48. Pc [Estimated regression model (A.23) vs. Simulation model] 


The estimated regression model (A.23) summarizes the simulation output well. 


However, we will use the simpler estimated model (A.18) rather than the estimated 
model (A.23). 
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Since we consider few alternative regression models which consider the effects of 
the MPA’s information retention (include the mean time until the MPA loses 
classification information of a classified vessel, 1/y as an independent variable), there 
may be an appropriate regression model in which 1/y is statistically significant to 
estimate the MIO capability ( P.) well. However, because the estimated regression model 
(A.18) summarizes the simulation output well, we consider the logistic regression model 


(A.18) to analyze the MIO capabilities ( P. ). 


2. Probability that the one R is detected, correctly classified and escorted 


before leaving the AOI, P, 
We consider the following two types of logistic regression models as alternatives. 


e 1/y is considered as a single independent variable. 
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When the sixth independent variable (the mean time until the MPA loses 


classification information of a classified vessel, 1/y) is 0, the model (A.24) corresponds 


to the model (A.19) which does not consider I/y. 
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e 1/y influences the number of White vessels in the AOL w. 
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When the mean time until the MPA loses classification information of a classified 


vessel, 1/y is 0 or n is large, the model (A.25) corresponds to the model (A.19) which 
does not include 1/w. n is determined so that the four independent variables are 


statistically significant in this regression model. 


The statistically fitted logistic regression models: 


e 1/y is considered as a single independent variable 
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(A.26) 


SUMMARY OUTPUT 


Regression Statistics 


Multiple R 1.000 
R Square 1.000 
Adjusted R Square 0.964 
Standard Error 0.022 
Observations 34 
ANOVA, 

df Ss MS F Significance F 
Regression 6 51.477 8.579 18264 0.000 
Residual 28 0.013 0.000 
Total 34 51.490 

Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 

bO 0.000 - - - - - 
b1 1.121 0.032 35.522 0.000 1.057 1.186 
b2 -1.474 0.036 -40.484 0.000 -1.549 -1,399 
b3 1.966 0.063 31.181 0.000 1.837 2.095 
b4 -2.797 0.158 -17.711 0.000 -3.120 -2.473 
b5 0.271 0.027 9.851 0.000 0.214 0.327 
b6 0.002 0.003 0.758 0.455 -0.004 0.009 


Since the p-value of b, is 0.455>0.05 and the 90% confidence interval contains 0, 
b, is not statistically different from 0. Figure 49 and Figure 50 display the outputs of the 


regression model (A.26) and those of the simulation model. 


Logit [Estimated regression model (A.26) vs. Simulation model] 
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Figure 49. Logit [Estimated regression model (A.26) vs. Simulation model] 


107 


P_ [Estimated regression model (A.26) vs. Simulation model] 
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Figure 50. Pg [Estimated regression model (A.26) vs. Simulation model] 


The estimated regression model (A.26) summarizes the output of the simulation 
model well. However, we employ the estimated model (A.19) rather than the estimated 
model (A.26) because the sixth independent variable of the model (A.26) is not 


statistically significant in the regression model. 


e 1/y influences the number of White vessels in the AOL, w. 
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SUMMARY OUTPUT 


Regression Statistics 


Multiple R 1.000 
R Square 0.999 
Adjusted R Square 0.965 
Standard Error 0.033 
Observations 34 
ANOVA 

df ss MS E Significance F 
Regression 5 51.458 10.292 9403 0.000 
Residual 29 0.032 0.001 
Total 34 51.490 

Coefficients Standard Error t Stat P-value Lower 95% Upper 95% 

bO 0.000 . . . . . 
b1 1.124 0.048 23.305 0.000 1.025 1.222 
b2 -1.495 0.057 -26.406 0.000 -1.611 -1.379 
b3 2.055 0.101 20.283 0.000 1.848 2.263 
b4 -3.031 0.256 -11.831 0.000 -3.555 -2.507 
b5 0.306 0.045 6.809 0.000 0.214 0.398 


Since the R-square value of the regression model (A.27) is 0.999, and the p-value 


of each independent variable is 0, the estimated model (A.27) summarized the simulation 


model output well. When n is less than 10, we could not find an appropriate logistic 


regression model which satisfies a high R-square value and the significance of the four 


independent variables. Figure 51 and Figure 52 display the outputs of the regression 


model (A.27) and those of the simulation model. 
Logit [Estimated regression model (A.27) vs. Simulation model] 
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Figure 51. Logit [Estimated regression model (A.27) vs. Simulation model] 
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P_ [Estimated regression model (A.27) vs. Simulation model] 
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Figure 52. P, [Estimated regression model (A.27) vs. Simulation model] 


The estimated regression model (A.27) summarizes the simulation output well. 
However, we will use the simpler estimated model (A.19) rather than the estimated 


model (A.27). 


Since we consider few alternative regression models which consider the effects of 
the MPA’s information retention (include the mean time until the MPA loses 


classification information of a classified vessel, 1/y as an independent variable), there 
may be an appropriate regression model in which 1/wy is statistically significant to 
estimate the MIO capability (P, ) well. However, because the estimated regression model 


(A.19) summarizes the simulation output well, we consider the logistic regression model 


(A.19) to analyze the MIO capabilities ( P, ). 
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APPENDIX 4 VALIDATIONS OF THE ESTIMATED 
LOGISTIC REGRESSION MODELS 


A. OBJECTIVE 

The objective of this appendix is to study the ability of the estimated logistic 
regression models of Chapter IV for the Maritime Intercept Operations (MIO) 
capabilities to predict simulation output with new/different input values which lie in the 
parameter intervals given in Table 8 on page 47. The ability of the estimated regression 


models to predict simulation output using parameters outside of the intervals is not 
studied. 


B. ESTIMATED LOGISTIC REGRESSION MODELS 
1. Probability that the one R is detected and correctly classified before 
leaving the Area of Interest (AOI), P. 
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2 Probability that the one R is detected, correctly classified and escorted 
before leaving the AOI, P, 
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C. INPUT DATA 

To design new input data set, we apply the similar approach by which we 
designed the original input data set. Design points (#1 — 17) are obtained from the NOLH 
based on Table 3 (page36), and design points (#18 — 34) are intentionally added to obtain 
relatively higher probabilities for P. and P, by using the other NOLH based on Table 5 


(page38). The design points of Table 26 are completely different from those of Table 4 
(page37); however they are chosen from the same parameter intervals given in Table 8 on 


page 47 as the original simulations. 


la sa Oy) a Sa Fl Ma al 
ID NM hrs 

Sas ae ee ee ee ee eee 
|____2|_376|_s6|_{s_§.3_0.931_0.63) _1} 
ee) | | | ee) 
| CAT] 10] 4] 45] 0.98 
L__§|_26]_ta{__197]_40)__ 0.98) 
a EE EEE Ee a aa 
Sit ist 
aa a ae ee ae ea) 
___.3} ao} sot] et 
0) =) © | = 2s) 2] ==) 
2 Saat 
= ee eee ee eee a ae eee 
| |<) = =) =) =e) =) 9 eZ) 
fe AB] G19 | 29 0) = 10.83] -—= 0.60) 28 
a |) ae ee ee eee ee ee eee 
aS ee”) ae ee Se an ee 
|_____19_226]_o}__ige|_&.3]_0.961__0.931_3. 


ealepien 
ao 


P00 


se 59] 2a 
| 9] 38] 815.9 
-__o|__200}_s{_9.4 


na 9] 16.3] 4 3] 980.97 
TZ: I Ed | 
| SA] 03] SA] tS] A] 7] St] 10) 


Table 26. Design of experiment 
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D. | COMPARISON (LOGISTIC REGRESSION MODELS VS. SIMULATION 
MODELS) 


1. Probability that the one R is detected and correctly classified before 
leaving the Area of Interest (AOI), P. 


Table 27 displays the outputs of the estimated logistic regression model (A.28) 


and those of the simulation model. Table 27 is sorted in descending order based on the 


outputs of the simulation model. 


Design] Mx | My | W v Vi u Crr | 1épsi eee model 
Point | (NM) | (NM) (kt) | (kt) | (kt) Ae in (hrs) Regression Modell = Po SY Logit 
ID 2 
Oo] 6 | 0.5875) 
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Table 27. Estimated regression model (A.28) and the simulation model 
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Figure 53 and Figure 54 display the outputs of the estimated logistic regression 
model (A.28) and those of the simulation model. The estimated regression model (A.28) 


summarizes the output of the simulation model well. 


Logit [Estimated regression model (A.28) vs. Simulation model] 
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Figure 53. Logit [Estimated regression model (A.28) and the simulation model] 
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Figure 54. Pc [Estimated regression model (A.28) and the simulation model] 
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2: Probability that the one R is detected, correctly classified and escorted 
before leaving the AOI, P, 


Table 28 displays the outputs of the estimated logistic regression model (A.29) 


and those of the simulation model. Table 28 is sorted in descending order based on the 


outputs of the simulation model. 


Mx | My Crr | 1épsi Estimated Simulation model 
(NM) | (NM) ie a a) te ‘lian (hrs) [Regression Model = Pe | Logit 
Lf peat | Pe Mean | m8 S68 | S68 
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Table 28. Estimated regression model (A.29) and the simulation model 
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Figure 55 and Figure 56 display the outputs of the estimated logistic regression 
model (A.29) and those of the simulation model. The estimated regression model (A.29) 
summarizes the output of the simulation model well. 


Logit [Estimated regression model (A.29) vs. Simulation model] 
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Figure 55. Logit [Estimated regression model (A.29) and the simulation model] 
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Figure 56. Pg [Estimated regression model (A.29) and the simulation model] 
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